Bring Your Own Python: Extending the METplus Forecast Verification System Capabilities for Users

The Model Evaluation Tools (METplus) verification system consists of a set of core tools (hereafter “MET”) written in C++, a suite of Python scripts to wrap the configuration and execution of those tools, and utilities for the aggregation and plotting of results. MET is generally designed to accept a forecast dataset and an observation dataset which are used to compute statistics and other measures of performance to quantify forecast skill. In nearly all cases, the forecast dataset is a two-dimensional gridded field of data while the observations can be either two-dimensional gridded data or a set of individual points defined by their latitude and longitude location and accompanying metadata. Historically, MET accepted gridded data in NetCDF files complying with the Climate and Forecast (CF) metadata conventions, or GRIB format and point observation data in several ASCII formats or the BUFR data format. Since version 8.0 of MET, the C++ code has been extended by enabling Python support at the time of compilation. This approach relies on including and linking to the library files of an existing Python installation at the time of MET’s compilation.

The support for embedding Python within MET has opened the door for users to incorporate novel file formats, input datasets, derived fields, and data manipulation with the entire Python ecosystem at their fingertips. The primary purpose of Python embedding has been to replace input data files with Python scripts that serve data instead. Gridded data, point observations, and pre-computed matched forecast and observation pairs can be supplied to the MET tools in this manner. Users can substitute a data file for a special keyword and the full path to their Python script along with any command line input arguments for their script. At runtime, the MET C++ code is initialized and then executes the user’s Python script instead of performing a data file read. Depending on whether the user is using the Python installation used to compile the MET C++ code or their own Python installation to run their script, data are either passed in memory directly to the C++ code or written to a temporary file and read in by the C++ code. Specific requirements exist for supplying gridded data, point observations, and matched pairs via Python. Many of these requirements concern metadata typically obtained from the input data file that in some cases the user must curate on their own. A gallery of user contributed use cases has been amassed featuring these capabilities to highlight the broad range of applications and assist other users with implementing them in their workflow. A detailed discussion of the implementation as well as a brief demonstration will be presented.