Comprehensive automated quality assurance of daily surface observations: the GHCN-daily example

This presentation will describe a comprehensive set of fully automated quality assurance (QA) procedures for observations of daily surface temperature, precipitation, snowfall, and snow depth. The QA procedures are being applied operationally to the daily Global Historical Climatology Network (GHCN) – Daily data set. Since these data are used for analyzing and monitoring variations in extremes, the QA system is designed to detect as many errors as possible while maintaining a low probability of falsely identifying true meteorological events as erroneous. The system consists of 19 carefully evaluated procedures that detect duplicate data, climatological outliers, and various inconsistencies (internal, temporal, and spatial). Manual review of random samples of flagged values is used to set the threshold for each procedure such that its false-positive rate is minimized. In addition, the tests are arranged in a deliberate sequence such that the performance of the later checks is enhanced by the error detection capabilities of those preceding them. Based on an assessment of each individual check and a final evaluation for each element, the system flags 0.24% of the data set, is estimated to have a false-positive rate of 1-2%, and is effective at detecting both the grossest errors as well as more subtle inconsistencies among elements.