Interpretations of Ensemble-Based Forecast Guidance from the 2017 Spring Experiment Survey

Operational and experimental convection-allowing model (CAM) ensembles and probabilistic forecast guidance are becoming increasingly available to forecasters. As a result, the paradigm for interpreting forecast guidance is evolving from one that is deterministic to one that is probabilistic. The strengths and limitations of CAMs are being tested and evaluated by model developers and users within the NOAA National Weather Service, National Centers for Environmental Prediction, and within NOAA testbeds. The focus of these evaluations is on verification statistics. While verification statistics provide information useful for evidence-based decision making, they do not provide insight into how probabilistic forecast guidance is interpreted or used by meteorologists.

The goal of this study is to sample and document meteorologists’ interpretations of probabilistic forecast guidance via a survey. This survey was administered to 62 meteorologists as a part of the first evaluation of the experimental Warn-on-Forecast (WoF) system within the 2017 Spring Experiment, held at the NOAA Hazardous Weather Testbed in Norman, Oklahoma
1 May – 2 June. The WoF system is a frequently updated, regional-scale, on-demand convection-allowing ensemble analysis and prediction system, nested with an hourly convection-allowing ensemble forecast system (currently HRRRe). This system produces 0–3-h predictions of individual convective storms and mesoscale environments that provide probabilistic forecast guidance, such as the probability of simulated reflectivity above a threshold at a grid point, and ensemble percentile values (e.g., 90^th) of accumulated rainfall, 2–5-km updraft helicity, and 0–2-km vertical vorticity, among others.

Meteorologists were presented 12 open-ended and multiple choice questions that queried their interpretation of probabilistic and percentile-based products in spatial, temporal, and severity space using ensemble forecast output from the WoF system. A few survey questions include: “In an ensemble-based probabilistic forecast, what do you think the 70^th percentile value of accumulated rainfall represents?” “Given the information presented, what is the probability of exceeding 0.5” of rainfall within box A?” Questions on the joint interpretation of probabilistic and percentile products are also included. The analysis of responses to these and the other questions is in progress and this presentation will report the findings.