Forecaster’s Mental Workload while Issuing Probabilistic Hazard Information (PHI) during the 2018 Hazard Services PHI Hazardous Weather Testbed

During spring 2018, as part of the FACETS project, a Hazardous Weather Testbed (HWT) was run to assess and advance the Probabilistic Hazard Information (PHI) concept. The Hazard Services (HS) experiment explored operational aspects of the PHI tool and paradigm. The testbed was run for 3 weeks and each week 2 National Weather Service forecasters were trained to use the HS PHI. Each week forecasters completed 6 hazardous weather scenarios. Each weather case had a duration of 2-3 hours. After each scenario, forecasters completed the NASA-Task Load Index (TLX) to evaluate their workload. In addition to scoring workload, forecasters also responded to each workload sub-dimension to provide contributing factors to workload. The six sub-dimensions of workload, mental demand, physical demand, temporal demand, performance, effort, and frustration are analyzed to determine significant contributing factors to workload.

The 2018 Hazard Services PHI workload average was 47.7 (out of 100, std dev 21.6, range 77.7). Significant contributors to workload include challenging meteorology, tornadic storms, knobology, system performance, collaboration, automation and the PHI paradigm.

Forecasters were presented with challenging meteorological scenarios to test the limits and capabilities of the PHI system. These challenging scenarios included many situations with tornadic development. The Hazard Services interface was new to many forecasters and it took some time to learn the new controls. Forecasters stated that as the week progressed, and they gained more experience with the tool, they became more comfortable with the controls to produce and manage PHI objects. System performance affected use of the new tool. When large number of objects were generated, the system often slowed down. The PHI system allows collaboration and hand-off of PHI objects across CWA borders. Forecasters were required to communicate and decide when to hand off and take control of PHI objects as they developed. Automated guidance helped forecasters triage their efforts and focus their interrogation as well as monitor widespread and low probability storms. Forecasters were challenged in determining probabilities of threats in the new paradigm, not only the current probability of a storm, but also how that probability develops over the life of the storm. Hazard services PHI was developed to test the operational capacity and limitations of the PHI paradigm. Forecasters worked scenarios in which they were responsible to issue either only tornado objects, only severe objects, or both severe and tornado objects. The average workload for severe only cases was 34.4, tornado only 60.9 and for both tornado and severe objects, 48.3.