First, we formulate a spectral budget that relates the evolution of the convective length scale to the vertically-integrated diabatic feedbacks. We link this budget to the moist static energy variance budget, widely used to diagnose diabatic feedbacks in cloud-permitting simulations. We then evaluate the radiative, convective and advective terms of this budget in a set of three-dimensional cloud-permitting simulations across a range of sea surface temperatures, using the System for Atmospheric Modeling. We find that the surface flux feedback (mostly latent heat flux) drives the convective cluster to a scale close to 1000km, while the radiative feedback (mostly longwave cloud feedback) stretches the convective clusters to larger scales, especially at low sea surface temperatures for which the radiative feedbacks are most intense. These first results explain why convective length scales are larger for low sea surface temperatures, and why the convective cluster size depends on the domain's size below 1000km. We run mechanism denial experiments to confirm our results, where the surface fluxes and/or the radiative cooling rates are homogenized across the horizontal domain. Our results underline the importance of observing and/or simulating surface fluxes, radiative and advective feedback across a wide range of spatial scales to understand the characteristics of turbulent moist convection.