Assimilation of All Sky Radiances from Research to Operations

Clouds are an essential component of the atmosphere as they occupy a central role in the earth system's radiative budget. A proper depiction of the cloud microphysical parameters is crucial for a wide range of aerospace, aeronautic and defense applications. However, clouds are still poorly represented in the initial conditions of state-of-the-art numerical weather prediction models. This results in major limitations for weather prediction, particularly for weather events that can be uniquely sensitive to moist dynamical processes in cloudy regions. Satellite radiance observations affected by clouds and precipitation are often discarded, which means that large quantities of potentially valuable data are lost. The assimilation of radiances in cloudy pixels requires overcoming difficult challenges such as the accuracy of the radiative transfer models due to scattering in cloudy conditions, the estimation of the model and observation uncertainty, the ability to handle strong non-linearities, the correction for biases in satellite observations and the adaptation for non-Gaussian error distributions and multiple spatio-temporal time scales.

The quality control procedures for microwave and infrared radiances in cloudy regions have to be updated to globally applicable methods that can be implemented in operations and do not require frequent tuning. This is a challenging problem since data assimilation can be strongly affected by outliers that reside in the tails of the observation-minus-background distribution. New norms such as the Huber norm or the Iteratively Reweighted Least Square Method (IRLS), which transition from an L2 norm in the middle to an L1 norm at the tails, have recently been introduced. Those norms give less weight to the tails of the probability density function, such that the minimization of the cost function is less affected by outliers. We implemented the IRLS norm as cost function of the Gridpoint Statistical Interpolation (GSI) system and conducted an assimilation experiment with GOES imager (channels 4 and 6) in which we relaxed the background check and cloud screening procedures in the observations quality control. The results show that the conditioning of the minimization problem was significantly affected. As one might expect, the choice of the parameter controlling the transition from the L2 to L1, is critical. We are working on setting an objective and automatic procedure to estimate this parameter from the innovations.

One concern with all-sky radiances is that bias predictors can absorb some useful signal and thus hide model deficiencies that should be corrected in the analysis. For this reason, we removed cloud liquid water from the list of bias correction predictors for all-sky radiances. We identified the interface between the model microphysics and the radiative transfer in the observation operator (CRTM) as a prime source of biases. Hypotheses about various particle shapes and radiative properties in the prediction model and forward operator do not necessarily correspond. For example, when the first-guess fields come from a WRF forecast using the Thompson microphysics scheme, we found that taking 20% of the forecasted snow and considering this as ice in the CRTM calculation significantly improves the fit to observations. We are now planning on a more systematic approach by considering this parameter estimation problem within the framework of the observation variational bias correction scheme. Feature calibration and alignment can also help to reduce displacement error.

The variational data assimilation system has the ability to handle non-linearities in the observation operator via multiple outer loops, which correspond to a sequence of linear minimizations with a re-linearization at each outer-loop level. As expected, the number of outer loops required to reach convergence is larger for infrared instruments compared to microwave. The radiative impact of clouds is more non-linear in the infrared band since scattering is dominant. Furthermore, we found that the number of iterations within the inner loop needs to be smaller for the infrared. Experiments show that it is possible to define a state region for the analysis increments, where the non-linear problem is assumed to be properly represented by its linearized counterpart. For infrared channels, we found that it was not needed to set the number of iterations beyond 20-30 iterations in each loop. We are currently investigating the use of retrievals to improve at a very affordable cost the quality of the first-guess. We expect to get the analysis to a more linear regime without needing too many outer-loops.

We added the cloud water mixing ratio and the ice water mixing ratio to the set of GSI control variables. In order to insure a multivariate analysis of the cloud hydrometeors we developed a standalone program to compute background error statistics of all the cloud microphysical variables (i.e. mixing ratios for cloud liquid water, cloud ice, snow and rain). This program works for both regional and global models. These statistics represent the input parameters for the static background error covariance matrix, which is modeled via a sequence of operators in the GSI variational system. Preliminary tests with the WRF regional model indicate that a full multivariate analysis of cloud parameters may require a hybrid assimilation approach and the choice of the proper hydrometeors for control variables is critical.

We present our recent progress toward the assimilation of all sky radiances with the GSI community system. This work primarily focuses on quality control, non-linearity and control variables. Other pressing issues that will need to be addressed in the near future include: the need of accurate surface emissivity and cloud properties for various instruments in CRTM, the detection and estimation of the inter-channel correlation caused by the presence of clouds, a physically consistent multivariate analysis so that the hydrometeors persist during the forecast ensuing the analysis, and a method and data to validate the assimilation of cloudy radiances.