Robust Statistical Quality Control for Atmospheric Radiosonde Data

As our knowledge of the earth and its atmosphere grows, so do the datasets that contain this information. The Upper Air Database (UADB), housed within NCAR's Research Data Archive (RDA), contains millions of atmospheric observations from radiosondes and pilot balloons. These observations are valuable as they serve as ground truth for satellite data, are the crux of numerical weather models, and are a critical foundation for atmospheric reanalyses. Although the spatial and temporal coverage of the archive is extensive, the observations have no measure that indicates how trustworthy they are. Proper quality control (QC) of this data has the potential to broadly impact many ongoing climate studies. The RDA support team is interested in providing archived datasets, such as the UADB, to the scientific community, which are enhanced by quality flags. With this in mind, we have adapted a statistically robust and automatic QC procedure with a high probability of detecting errors and a low probability of misidentifying true values as errors in a historical radiosonde temperature record. A previous method utilized Lanzante's biweight estimator of mean and standard deviation (Durre et al, 1996) to QC mandatory levels in historical thermal data using z-scores in a two-step method. We optimize their method by using Huber's robust M-estimator. This provides an asymmetric estimate of standard deviation and detects errors with higher accuracy. Additionally, we derive statistics for the QC of significant levels by spline interpolation with respect to the logarithm of the mandatory pressure levels. Our procedure is applied to the entirety of NCAR's UADB thermal, geopotential height, and moisture data, and quality flags and z-scores will be made available to users. We will present illustrations that show the robustness of this method and highlight the data findings when it is applied to the UADB at NCAR.