The results show that the budgets of the normal components of the SGS stress have more complex behaviors in unstable surface layers than in neutral surface layers due to the complex interactions among shear, buoyancy, pressure, and the presence of the ground. For neutral surface layers, energy gained by τ11 from the mean flow is fed to τ22 and τ33 by the pressure-strain term, which diminishes with decreasing filter size as smaller eddies are increasingly more isotropic. For unstable surface layers, energy is gained by both τ11 and τ33. For large filter scales, the pressure-strain term feeds energy from τ33 to τ11 and τ22 because the vertical motion of the large convective eddies are blocked by the ground. Thus, for large filter scales, the pressure-strain term contributes to the anisotropy of the SGS stress. As the filter scale decreases, the effects of the ground and buoyancy diminish while the anisotropy in the SGS stress becomes relatively more important. Here, the pressure-strain term reverses role and now acts to re-distribute energy from the horizontal to the vertical velocity component, driving the SGS stress tensor toward isotropy. For very small filter scales, the SGS stress becomes increasingly more isotropic, therefore the pressure strain term diminishes. These results have strong implications for using model transport equations for SGS stress, particularly in correctly parameterizing the pressure-strain-rate correlation.