Development of a Tabletop Method to Refine Products and Improve NWS Impact-Based Decision Support Services

The National Weather Service (NWS) is tasked through the Weather Research and Forecasting Innovation Act of 2017 with providing “...relevant information and interpretive services to enable core partner’s decisions...” that ensure public safety, in both episodic (event-driven) and routine (ongoing) ways. The first of four core partner categories is the emergency manager (EM), meaning local, state, federal, tribal, or territorial public safety officials who protect the public from weather hazards and weather-impacted hazards. Prior research has begun to identify how EMs receive, understand, and use NWS Impact-Based Decision Support Services (IDSS), and how the NWS Weather Forecast Offices (WFO) come to understand the needs of their core partners. In addition, most NWS WFOs conduct Integrated Warning Team (IWT) meetings with their core partners to build relationships, share information, and discuss problems of mutual interest. As the state of forecasting science improves and new tools such as probabilistic outputs from ensemble models become available, WFOs need to be able to rapidly test and refine innovations in decision support and collect feedback to assure they meet user’s decision needs.

This study aims to further develop a specific type of IWT meeting in the form of a tabletop exercise created by the NWS Amarillo and a regional emergency management response agency. In so doing, our research team conducted exercises with five geographically distributed WFOs. The exercises took three forms: 1) individual interviews with end-users after real-time weekly and special event webinars for the CWA, 2) a tabletop exercise and group discussion with a subset of users after a simulated event-based webinar for the CWA, and 3) a tabletop exercise and group discussion after a simulated briefing that was targeted to a particular county or location.

Both the method of running the exercise and the data from each exercise were studied. After each exercise, we examined the efficacy of the method, particularly, whether discussion questions were effective in eliciting substantive, on-topic feedback. For example, some questions used in the first three exercises yielded the information sought from participants and some were taken in multiple directions, as previously reported. Those findings will be expanded with analysis from the last two exercises and a final method will be described. The remainder of the talk will address the findings from the exercises. Of special note, exercises included probabilistic graphics such as 10th and 90th percentile rainfall amounts, box-and-whisker plots for rainfall at a particular site, and probability of exceedance of varying amounts of rainfall at a particular site(s) on either a stacked histogram or map. End-users were also asked about their decision processes. For example, participants asked if charts showing 24-hour time periods could instead show greater precision because 24-hr periods do not align with their decision-making needs. Analysis triangulated end-user needs with end-user interpretation and reported use of the graphics, which resulted in recommendations to each WFO on how to improve their briefings.