Comparison of traditional and neighborhood verification statistics for the Hydrometeorology Testbed Forecast Demonstration Project using the Model Evaluation Tools (MET)

The National Oceanic and Atmospheric Administration (NOAA) Hydrometeorology Testbed (HMT) is a forecast demonstration project aimed at accelerating the infusion of new technologies, models, and scientific results from the research community into daily forecasting operations of the National Weather Service and its River Forecast Centers (RFCs). Advanced observational and modeling tools for quantitative precipitation estimation (QPE) and quantitative precipitation forecasting (QPF) are used to improve hydrologic forecasts and warnings. The HMT-West regional testbed has been operating over the American River Basin in California during the cool season since 2006. The 2010 field campaign includes the production of a 9-member ensemble forecast based on the Weather Research and Forecasting model.

The Developmental Testbed Center (DTC) has partnered with HMT to perform objective verification of its ensemble forecasts. The DTC has developed the Model Evaluation Tools (MET) verification package through the generous support of the U.S. Air Force Weather Agency and NOAA. The HMT-West ensemble is initialized every six hours and produces 3-hourly forecasts out to five days. MET is run twice daily on each ensemble member, the ensemble mean, and the Global Forecast System (GFS) model, interpolated to the HMT-West domain, to verify 6-hour and 24-hour QPF. The 6-hour and 24-hour QPF is compared to the StageIV QPE product. The QPF versus StaveIV comparison is performed using the MET Grid-Stat tool to compute traditional and neighborhood verification statistics. Statistics are computed over five hydrologic units in California as well as the California-Nevada RFC and the entire HMT-West domain.

This paper analyzes the MET output from the 2010 HMT-West field campaign and compares the traditional verification scores derived from contingency tables using several precipitation thresholds to the neighborhood verification scores using the same thresholds and various neighborhood sizes. In particular, the neighborhood Fractions Skill Score (FSS) and Fractions Brier Score (FBS) are used. The enhancements to the HMT-West evaluation through the inclusion of the FSS and FBS are shown.