Herbie: A Python Package for Accessing Numerical Weather Prediction Data

Herbie is a python package that downloads recent and archived numerical weather prediction (NWP) model output in GRIB2 format from different archive sources (i.e., NOMADS, AWS, GCP, Azure, etc.) and reads the data with xarray via cfgrib. Open data programs, such as the NOAA Open Data Dissemination (NODD) program, has made NWP output more accessible than ever before. Data discovery tools, like Herbie, bridge the gap between the data source and the software used to process the data, making users more productive with the data. The Naval Research Laboratory uses Herbie to access these various datasets for model intercomparisons.

Herbie was first developed to access archived High-Resolution Rapid Refresh (HRRR) model data but has been extended to access other NWP output including RAP, GFS, GEFS, ECMWF, RTMA, HRDPS, and other open datasets. Herbie removes the burden of data acquisition from the user. The driving philosophy behind Herbie is that a scientist shouldn’t worry about where model data is archived, just so long as it accessible (a known URL) and in a usable format (GRIB2). Since NWP output files are often large (hundreds of megabytes), Herbie exploits a desired variable’s known byte ranges to download partial files. In this talk, I will give an overview of Herbie and encourage the use of tool like GitHub in research workflow.

Distribution Statement A. Approved for public release. Distribution is unlimited.