BUFR2NetCDF – Converting Observational Data to a Self Describing Archive Format

The majority of observational data collected and distributed by the World Meteorological Organization (WMO) Global Telecommunication System (GTS) are done so via the BUFR data format. There are many good reasons for this, such as the ability to store nearly any observational data type, flexibility for missing/unused parameters, and file compressibility, in other words BUFR is a very good transport container. BUFR data are a table driven data format, meaning that a separate table is maintained for the encoding/decoding of the data stored within. The WMO, as well as many other operational data centers such as the National Oceanic and Atmospheric Administration's (NOAA) National Center for Environmental Prediction (NCEP), maintain the metadata tables for storing and retrieving data within a BUFR file. Often the table data is not embedded with the BUFR files (though NCEP does embed the tables) and it can be challenging for the user to extract the table metadata, or locate the proper version of the table to obtain the needed metadata for a BUR file. More generally, non-expert users find BUFR a difficult format to parse, and to use and understand correctly. This presentation introduces a tool for converting BUFR data files to the self-describing NetCDF format. Of note is that the resulting NetCDF file will incorporate the new Discrete Sampling Geometries of the Climate and Forecast (CF) metadata convention. This will provide users of archived observational data: greater ease of use, assurance of data and metadata integrity for their research, and improved provenance.