Fat Tail Probability Distributions for Variational Quality Control of All-Sky Radiances

Clouds are an essential component of the atmosphere as they occupy a central role in the earth system’s radiative budget. A proper depiction of the cloud microphysical parameters is crucial for a wide range of aerospace, aeronautic and defense applications. However, clouds are still poorly represented in the initial conditions of state-of-the-art numerical weather prediction models. This results in major limitations for weather prediction, particularly for weather events that can be uniquely sensitive to moist dynamical processes in cloudy regions. Satellite radiance observations affected by clouds and precipitation are often discarded, which means that large quantities of potentially valuable data are lost. The quality control procedures for microwave and infrared radiances in cloudy regions have to be updated to globally applicable methods that can be implemented in operations and do not require frequent tuning. This is a challenging problem since data assimilation can be strongly affected by outliers that reside in the tails of the observation-minus-background distribution. New norms such as the Huber norm or the Iteratively Reweighted Least Square Method (IRLS), which transition from an L2 norm in the middle to an L1 norm at the tails, have recently been introduced. Those norms give less weight to the tails of the probability density function, such that the minimization of the cost function is less affected by outliers. We implemented the IRLS norm as cost function of the Gridpoint Statistical Interpolation (GSI) system and conducted an assimilation experiment with GOES imager (channels 4 and 6) in which we relaxed the background check and cloud screening procedures in the observations quality control. The results show that the conditioning of the minimization problem was significantly affected. As one might expect, the choice of the parameter controlling the transition from the L2 to L1, is critical. This presentation focuses on reporting work toward setting an objective and automatic procedure to estimate this parameter from the innovations.