A Standards-based Approach to Data Encoding for Statistical Post-processing of Model Output

NOAA’s vision for modeling is to engage with the public, private, and academic sectors in a Modeling Community, and we are pursuing this goal aggressively. This vision extends beyond Numerical Weather Prediction (NWP) to Statistical Post-processing (StatPP) of NWP output. It also extends beyond the US borders as we engage our international partners, as appropriate. NOAA’s Next Generation Global Prediction System (NGGPS) has formed a Working Group on Post Processing to pursue the StatPP aspects. At the same time, the NWS is in the early stages of developing a Weather Information Statistical Post-processing System (WISPS). WISPS is a community-based software system, designed to support StatPP using modern software languages and self-describing data formats.

The netCDF (Network Common Data Form) data storage standard and its associated Climate and Forecast conventions (CF; netCDF-CF) have served the atmospheric science community quite well for decades. Many organizations have applied these frameworks to the problem of StatPP. Unfortunately, these organizations have settled on a variety of solutions with limited success in interoperability. This presentation will propose an application profile of netCDF-CF that can address these interoperability problems. We do not expect these standards to replace GRIB2 (FM 92 GRIB Edition 2) and BUFR (Binary Universal Form for the Representation of meteorological data) as the data encoding methods of choice for the exchange of operational meteorological data between meteorological services. Rather, we expect these standards to enhance data archival and sharing throughout the weather enterprise. Compliant data files will also contain sufficient metadata to document the specific StatPP processes used in their generation.

This presentation will propose conventions for encoding the binary data of WISPS as well as introducing tools that facilitate the use of these conventions. The proposal includes a standardized vocabulary, and techniques to represent it in metadata. We emphasize the use of pre-existing standards from the International Standards Organization (ISO), Open Geospatial Consortium (OGC), and the World Meteorological Organization (WMO). Since these efforts focus on ancillary variables and metadata, the techniques can be applied to HDF and other data-encoding tools.