Machine Learning to Improve Sounding Retrievals in Planetary Boundary Layer

In recent decades, spaceborne microwave and hyperspectral infrared (IR) sounding instruments on Aqua and Suomi NPP have led to significant enhancements in the accuracy of numerical weather prediction (NWP). Sounding retrieval algorithms reconstruct a three-dimensional distribution of atmospheric temperature and water vapor from the observations, which consist of upwelling thermal emission and scattered radiance. A key challenge to many retrieval algorithms is the complexity of physically modeling these observations (particularly interesting phenomena such as severe storms and temperature inversions), which often depend on the geophysical quantities of interest in an indirect, nonlinear, and/or non-Gaussian manner. Statistical retrievals such as neural network and regression approaches can often overcome this challenge by learning the empirical relationship between the observations and a ground truth training set describing geophysical variables of interest. Artificial neural networks are particularly capable at learning and representing this empirical relationship, complexities and all, and, with a sufficient number of hidden units, are universal function approximators.

The introduction of Version 6 of the AIRS/AMSU Level 2 retrieval algorithm has improved accuracy and yield in cloud-covered scenes versus previous versions, further enhancing the utility of the retrievals. The improvements are attributable in part to the incorporation of a new first guess, the Stochastic Cloud Clearing/Neural Network (SCC/NN) algorithm, a statistical technique for performing temperature and water vapor retrievals which replaces linear regressions used in the previous versions. SCC/NN was first developed under a 2009 Science of Terra and Aqua award. Recently, we adapted SCC/NN for Cross-Track Infrared Sounder (CrIS) and Advanced Technology Microwave Sounder (ATMS).

Recent investigations suggest that both AIRS/AMSU retrievals and ECMWF ERA interim reanalysis remain challenged by boundary layer inversion features. Dedicated radiosonde measurements from the Marine ARM GPCI Investigation of Clouds (MAGIC) campaign in the northeast Pacific showed that both ECMWF and AIRS Level 2 retrievals tend to produce errors near sharp inversions, with AIRS smoothing over the vertical features and ECMWF placing them at the wrong level. Improved representation of boundary layer inversions would enhance understanding of important cloud processes in weather and climate. Boundary layer inversions, in general, are difficult to directly observe with AIRS and CrIS, due to instrument vertical resolution. In current AIRS/AMSU Level 2 retrievals, boundary layer phenomenology is mostly introduced via the SCC/NN first guess, which is trained with ECMWF profiles. Due to the limited direct observability of boundary layer features, machine learning approaches such as neural networks are uniquely well-suited to discovering and reproducing complex or indirect empirical relationships between the boundary layer features and the observations.

Recently, we have begun a new investigation funded by NASA's Resarch Opportunities in Earth and Space Science (ROSES), addressing significant changes to the NN retrieval methodology designed to improve representation of boundary layer inversions. This will be enabled by three significant upgrade steps:

• Per-retrieval covariance estimation and error prediction
• A second-pass NN to refine boundary layer inversions, trained on high-quality sonde data in the PBL
• Improved utilization of the horizontal and spectral resolution of the AIRS and CrIS instruments

We present our progress made to date on development and implementation of this algorithm.