The NOAA Testbeds & Proving Grounds: Advancing probabilistic hazards forecasting for decision support

NOAA's Testbeds (TBs) and Proving Grounds (PGs) are crucial mechanisms for transitioning research into operations within NOAA and other partner agencies, and ultimately into societal benefits. The thirteen TBPGs all focus on advancing probabilistic hazards forecasting for decision support in various ways, via testing and the evaluation of new and innovative forecasting and communication techniques, then facilitating the orderly transition of research innovations into operational implementation.

This presentation will discuss and give examples of the ways that TBPGs are working to advance probabilistic hazards forecasting, illustrated by specific examples from individual TBPGs or collaboration among them. These methods include first testing, evaluating, and refining the emerging innovations that are at the forefront of forecasting a spectrum of environmental threats, especially new probabilistic guidance and communication techniques. Evaluation of the UFS has been a focus across several testbeds over a number of years. For example, beginning in 2017, HWT, and in 2018, HMT have evaluated FV3 CAMS runs in a series of Spring Forecast Expts and Flash Flood and Intense Rainfall Experiments, culminating in a White Paper on the RRFS. HMT has evaluated RRFS in the Weather Prediction Center’s Excessive Rainfall Outlook, a probabilistic product. The Aviation Weather Testbed (AWT) work to evaluate how the probabilistic ensemble RFFS compares to HREF for the convective guidance suite for use in AWC Traffic Flow Management Convective Forecast (TCF) production process.Other activities include Climate TB’s work to extend UFS prototypes into ensembles to enable probabilistic forecasting, and the Coastal and Ocean Modeling TBs efforts to improve data assimilation algorithms for U.S. West & East coastal ocean forecast systems.

Second, they assist developers in the design and development of products to communicate risk in order to advance actionable forecasts, outlooks, watches, and warnings. This includes translating hazards into impacts, optimizing watches/warnings, and facilitating social science studies to understand how these products are interpreted and communicated. Examples include the Hydrometeorology Testbed’s (HMT) work to enhance a probabilistic Winter Storm Severity Index, and HWT contributin to the development of Warn on Forecast techniques.

Third, TBPGs put innovations in front of NOAA forecasters and users, including emergency managers and broadcast meteorologists, to introduce new products and get feedback on their utility to support operations. Over 500 individuals a year are involved in activities across TBPG experiments. For example, forecasters and others are involved in the experiments run by the Aviation Weather Testbed (AWT), Hazardous Weather Testbed (HWT), Space Weather Testbed (SWxTB), HMT, and the Hurricane and Ocean Testbed (HOT). These activities also serve to socialize emerging product types to the Weather Forecast Office network.

Finally, TBPGs provide feedback from operations and testing into research and research priorities; for example, the Developmental Testbed Center regularly iterates and provides feedback to research communities on issues with methods discovered during testing, which often results in research to solve those issues. The NWS Operations PG (OPG) tests systems, processes, and tools that will be used in NWS Weather Forecast Offices (WFOs), and evaluates what training may be needed before advances become used operationally in WFOs. TBPGs have worked collaboratively to demonstrate new tools, evaluate forecasting techniques, and understand the impact of mesoanalysis on forecast operations. For all of these, testing and evaluation are iterative, with innovations often going through several rounds of testing and refinement in experiments. Efforts are often collaborative between two or more TBPGs, for instance, HWT and OPG evaluated the collaborative forecast process (CFP) workflow for the new Warn-on-Forecast System, ProbSevere and Prob Tor. HMT and OPG worked together on evaluating the NWS Collaborative Forecast Process workflow for rainfall products.

TBPGs support the R2O process through working relationships for developmental testing, in quasi-operational frameworks among researchers and operational scientists/experts (measurement specialists, forecasters, IT specialists, etc.) including partners in academia, the private sector, and government agencies, aimed at solving operational problems or enhancing operations, in the context of user needs. TB activities are two-way interactions between developers and end users helping to facilitate both R2O and O2R, and are iterative, in which any particular project is generally tested multiple times and ways.
This presentation also will give an overview of this network and how it operates, some recent accomplishments, and upcoming challenges.