Automated processing and quality control of NCEI's Hourly Precipitation Dataset

The NOAA/NWS Fischer-Porter (F&P) weighing bucket precipitation gauge network consists of approximately 2000 stations that comprise a subset of the NWS Cooperative Observers Program network. For several decades it served as the primary source for NCEI’s DSI-3240 dataset, known as the Hourly Precipitation Dataset (HPD). The lengthy record of this dataset of hourly precipitation amounts, combined with its relatively high spatial density, provides an important source of data for hydrometeorological applications, supporting water resource decision makers, engineering, and weather forecasting including flood warnings. In the mid-2000’s, the NWS began upgrading the F&P stations by replacing punched paper tape with a digital recorder at most sites. As part of the effort to modernize the network, NCEI initiated an upgrade of its end-to-end processing and quality control system for the HPD data. This involved a change from a largely manual review and edit process to a fully automated system that removes the subjectivity that was previously a necessary part of quality control and processing. The conversion from paper to digital recording, along with development of an objective quality control process, is aimed at greatly improving the quality of the HPD data. A description of this automated system is provided. Included is an overview of the automated filtering and conversion from 15-minute gauge depth to incremental precipitation, development of hourly quality control algorithms using methods established for NCEI’s GHCN-Daily dataset, and the results of testing and validation. A beta release of this dataset was made available in March 2017, and a full operational release is forthcoming.