The results show that the budgets of the normal components of the SGS stress have more complex behaviors in unstable surface layers than in neutral surface layers due to the complex interactions among shear, buoyancy, pressure, and the presence of the ground. For neutral surface layers, energy gained by τ_{11} from the mean flow is fed to τ_{22} and τ_{33} by the pressure-strain term, which diminishes with decreasing filter size as smaller eddies are increasingly more isotropic. For unstable surface layers, energy is gained by both τ_{11} and τ_{33}. For large filter scales, the pressure-strain term feeds energy from τ_{33} to τ_{11} and τ_{22} because the vertical motion of the large convective eddies are blocked by the ground. Thus, for large filter scales, the pressure-strain term contributes to the anisotropy of the SGS stress. As the filter scale decreases, the effects of the ground and buoyancy diminish while the anisotropy in the SGS stress becomes relatively more important. Here, the pressure-strain term reverses role and now acts to re-distribute energy from the horizontal to the vertical velocity component, driving the SGS stress tensor toward isotropy. For very small filter scales, the SGS stress becomes increasingly more isotropic, therefore the pressure strain term diminishes. These results have strong implications for using model transport equations for SGS stress, particularly in correctly parameterizing the pressure-strain-rate correlation.