Objective Verification of Experimental Guidance in FFaIR and the WPC Watch Collaborator Forecast for the 2016–2017 Season

The 5th Annual Flash Flood and Intense Rainfall (FFaIR) Experiment was held at the Weather Prediction Center (WPC) Hydrometeorology Testbed (HMT) in College Park, MD over four weeks during the period from June 19 to July 21. The FFaIR Experiment brings together participants from across the weather enterprise to simulate a pseudo-operational environment in an effort to create experimental probabilistic forecasts and evaluate emerging models, tools, and datasets with the ultimate goal of improving flash flood forecasting. During the experiment, participants utilized and evaluated the latest experimental deterministic convection allowing models (CAMs) and CAM ensemble systems.

The WPC Watch Collaborator tool is informed by the Probability Winter Precipitation Forecast (PWPF) but applies National Weather Service (NWS) Weather Forecast Office (WFO) snow warning criteria for the entire CONUS. The tool highlights areas where there is some probabilities of a winter storm watch needed based on snow and ice criteria for both 12 and 24 hour amounts.

A component of the 2017 experiment was objectively evaluating quantified precipitation forecasts (QPF) from CAM guidance in the day two and day three timeframe. The objective analysis was performed using the Model Evaluation Tool (MET) and Method for Object-Based Diagnostic Evaluation (MODE) package, which generated both grid statistics and object based analysis for the experimental guidance. Verification results will be presented using Roebber Performance diagrams and select MODE imagery including a case study of CAM performance during Tropical Storm Cindy. MODE Analysis will also be shown, and will speak to object based statistics including angle, area ratio and centroid distance.

In addition to the FFaIR guidance, the WPC Watch Collaborator for the 2016-2017 winter season was also evaluated using MET and MODE. An analysis of MODE objects and reliability diagrams at various percentiles will be shown in effort to communicate forecast performance in terms of reliability, grid statistics and object based statistics (including angle, area ratio, and centroid distance).