Forecasters' Mental Workload while Issuing Probabilistic Hazard Information (PHI) during 2016 FACETs PHI Hazardous Weather Testbeds

Forecasters’ Mental Workload while Issuing Probabilistic Hazard Information (PHI)

during 2016 FACETs PHI Hazardous Weather Testbeds

 

 

Chen Ling¹, Joseph J. James¹, Seyed M. Miran¹ , Greg Stumpf ^2,3, Tracy Hansen⁴, Kevin Manross^5,4,
James Ladue⁶, Alyssa Bates^2,6, Christopher D. Karstens^2,7, Kristin Calhoun^2,7, James Correia^2,8,
Tiffany Meyer^2,7, Alan Gerard⁷, Lans Rothfusz⁷

¹Department of Mechanical Engineering, University of Akron, Akron, OH

²Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, OK
³NOAA/National Weather Service/Meteorological Development Laboratory, Silver Spring, MD
⁴NOAA/ESRL /Global Systems Division, Boulder, CO
⁵Cooperative Institute for Research of the Atmosphere, Colorado State University, Fort Collins, CO
⁶NWS Warning Decision Training Division, Norman, OK
⁷NOAA/OAR/National Severe Storms Laboratory, Norman, OK
⁸ NOAA/Storm Prediction Center, Norman, OK

During Spring 2016, two “Forecasting a Continuum of Environmental Threats” (FACETs) Probabilistic Hazard Information (PHI) Hazardous Weather Testbed (HWT) experiments were run to test and improve the Probabilistic Hazard Information (PHI) concept-The Hazard Services-PHI (HS-PHI) experiment and the PHI-prototype experiment. During each week of the 3-week experiment, two to three forecasters from the National Weather Service were trained to use either the HS-PHI system or the PHI-prototype tool to issue probabilistic hazard information on severe weather threats. They worked both archived weather scenarios and live cases of 2-3 hours duration. After each work session, forecaster’s mental workload was evaluated using NASA-Task Load Index (TLX) questionnaire. In this study, we summarize the trends of mental workload experienced by forecasters while issuing PHI in both experiments, and discuss the top contributing factors for increased workload.  We summarize the driving factors behind each of the six sub-dimensions of mental workload, namely mental demand, physical demand, temporal demand, performance, effort, and frustration.

The mean mental workload experienced during the Hazard Services-PHI HWT was 49.9 (out of 100, std: 23.9) with a range of 86.5. The most mentioned contributing factors for increased workload in HS-PHI were software errors, communication regarding CWA borders, number of objects/storms to keep track of, and the new paradigm change. For the PHI prototype, the mean mental workload was 46.6 (out of 100, std: 19), with a range of 70.8. The most mentioned contributing factors for increased workload in the PHI-prototype included learning to use automated guidance, number of objects/storms to keep track of, multiple displays, and formulating probabilities.

Throughout both HWT experiments, forecasters explored creating and managing PHI objects. In both experiments, forecasters mentioned formulating PHI in the new paradigm and keeping track of large number of storms/objects as top contributing factors for increased mental workload.  The paradigm shift challenged forecasters to explore how to communicate probabilistic information on severe weather threats. Forecasters reported increased mental workload resulting from creating probability trends, duration, confidence, and text products for each threat object. To maintain high performance, forecasters were concerned with ability to accurately and completely monitor all threats, while creating PHI objects for newly developing threats, and providing updates for continuously evolving threats.

Because of the differences in experimental setup between the two HWT experiments, different factors were also mentioned as the top contributor to mental workload.  In the HS-PHI experiment, the PHI tool was being implemented into AWIP II system, and the software errors and processing speed were major contributors to increased effort. The concept of collaboration across CWA borders was heavily tested in this experiment. Therefore, both software errors and communication regarding county warning area borders were mentioned as top contributors to mental workload. In the PHI-Prototype experiment, the PHI-prototype system was built on a web-based platform.  The experiment used two displays - forecasters interrogated radar data on one display with AWIPS2, and created and managed PHI objects on the second display using a Web browser. The PHI prototype implemented recommenders for probability suggestions and object outline suggestions. Forecasters also communicated with emergency managers and a TV forecaster in separate rooms to provide decision support. Therefore, working with the automated object generation and multiple displays were mentioned as top contributors.