Recent Advances in AERI Data Processing: Real-time Quality Control and Real-time Thermodynamic Retrievals

The Atmospheric Emitted Radiance Interferometer (AERI) is an automated ground-based spectrometer measuring downwelling thermal infrared radiance from the atmosphere. The instrument was originally developed by the University of Wisconsin-Madison Space Science and Engineering Center (SSEC) in the 1990s, and is now licensed for commercial distribution by ABB Inc. For over two decades, the AERI has been deployed extensively worldwide by the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) program as well as various research groups. Applications of AERI observations include thermodynamic profiling, as well as studies of cloud properties, land and sea surface emissivity, and spectroscopy.

Up to this point the AERI has been a research-grade instrument, requiring specialized training to operate, and in particular to analyze the data. The main output data product of the instrument is calibrated infrared spectral radiance, which is not a product that most meteorologists and atmospheric scientists can intuitively understand and analyze. We present two major advances in AERI data processing that address these shortcomings.

1. Real-time quality control (QC). A fraction of the data from any observing system can be compromised by single event upsets, transient environmental forcing, or fault conditions that need to be corrected. In the case of the AERI, high-level QC is performed by experts for the data in the ARM archive, but otherwise investigators are responsible for applying QC to AERI data on their own. This severely limits the accessibility of the dataset. To solve this problem, we have created a new algorithm to provide QC assessment of each new data record locally on the instrument in real-time. The method uses the extensive housekeeping data collected by the AERI as well as radiance products to perform an array of tests of the instrument state, in order to ascertain the data quality of each data record. This new tool not only helps in quickly identifying instrument issues, but it now allows even novice AERI users to confidently analyze AERI data, knowing that the data quality has been thoroughly vetted by the instrument experts.

2. Real-time thermodynamic retrievals. While there have been several retrievals for AERI data in the past, most had some severe limitations in their utility. We have recently developed a new retrieval called AERIoe (based on an Optimal Estimation method). AERIoe can be tuned with climatology specific to the observing site, and it can produce accurate profiles of temperature and water vapor in the lower troposphere, as well as retrieve cloud property and trace gas information. It operates under all sky conditions, providing profiles up to the cloud base height. AERIoe can also incorporate datastreams from other observing systems in order to further increase retrieval accuracy. Furthermore, as the retrieval uses a 1-dimensional variational method, a full error characterization is provided for each retrieved profile.

With these two major advances, the operational data product of the AERI becomes real-time accurate and quality controlled thermodynamic profiles. In essence, this is analogous to producing radiosonde profiles of temperature and water vapor in the lower troposphere every 20 seconds (albeit at a lower vertical resolution). These developments transform the AERI into a vastly more valuable tool for the meteorological and atmospheric science community. These advances should open up many new uses for AERI data, in particular for meteorological applications. Early results from field campaigns with networked deployment of up to 8 AERI instruments show promise in improving the performance of numerical weather prediction models and severe weather forecasts.