James Ladue6, Alyssa Bates6, Christopher D. Karstens2,7, Kristin Calhoun7, James Correia8, Tiffany Meyer7, , Alan Gerard7, Lans Rothfusz7
1Department of Mechanical Engineering, University of Akron, Akron, OH
2Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, OK
3NOAA/National Weather Service/Meteorological Development Laboratory, Silver Spring, MD
4NOAA Global Systems Division, Boulder, CO
5NOAA/ESRL/GSD, Evaluation and Decision Support Branch, Boulder, CO
6OU CIMMS/NWS Warning Decision Training Division, Norman, OK
7NOAA/OAR/National Severe Storms Laboratory, Norman, OK
8 NOAA/Storm Prediction Center, Norman, OK
During spring 2017, as part of the Forecasting a Continuum of Environmental Threats (FACETS) project, two Hazardous Weather Testbed (HWT) were run to assess and advance the Probabilistic Hazard Information (PHI) concept. The PHI prototype experiment analyzed new aspects and features of the PHI tool. The Hazard Services (HS) PHI experiment explored operational aspects of the PHI tool and paradigm. Each testbed was run for 3 weeks and each week 2 to 3 National Weather Service forecasters were trained to use the HS-PHI tool or the prototype PHI tool. Each week forecasters completed 3 real-time and 3 archived hazardous weather scenarios. Each weather case had a duration of 2-3 hours. After each scenario, forecasters completed the NASA-Task Load Index (TLX) mental workload survey to evaluate their workload. In addition to scoring workload, forecasters also described each workload sub-dimension in terms of contributing factors to workload. The six sub-dimensions of workload including mental demand, physical demand, temporal demand, performance, effort, and frustration are analyzed to determine significant contributing factors to forecaster’s mental workload.
The average mental workload experienced by forecasters in the PHI prototype testbed was 53.7 (out of 100, std: 14.2, range: 53.5). The greatest contributing factors were large number of objects, rapid storm development and pulse storms, new hazard (lightning), switching between Advanced Weather Interactive Processing System (AWIPS) system and PHI tool, and update frequency of object information. For the Hazard Services PHI testbed, the average mental workload was 58.2 (out of 100, std: 15.3, range 58.8). The most significant contributing factors affecting workload were the large number of objects (especially in Quasi-Linear Convective System (QLCS) events), software slowdowns, meteorological conditions, knobology, determining probabilities in the new paradigm and collaborating with neighboring County Warning Areas (CWA).
In both Hazard Services PHI and PHI prototype testbed, forecasters utilized the new method of communicating hazardous warning information using the PHI paradigm. A common contributing factor to workload in both experiments was a large number of PHI objects. Forecasters strived to keep all objects updated with accurate information and allow time for radar interrogation. An increased number of objects decreased the time forecasters had for radar interrogation and resulted in increased time between updates and ultimately loss of situational awareness. Another common factor between both experiments was rapid storm development, included pulsing storms. Automated guidance would sometimes lag behind storm development. Forecasters used the automation as a first guess and an aid to guide their interrogation focus.
In the PHI Prototype testbed, forecasters worked with a web-based PHI prototype tool on a separate display. Switching back and forth between displays resulted in increased workload. Forecasters also worked with automated guidance. When the guidance did not accurately follow storm development, forecasters were required to manually take over objects. Forecasters were also forecasting and using guidance for lightning threats. Lightning threats are not specifically forecasted in the current paradigm, so communicating specific threat information challenged forecasters.
Hazard services PHI system was developed to test the operational capacity and limitations of the PHI paradigm. Several factors were mentioned as significant contributors. Forecasters were challenged in formulating probabilities in the new paradigm, not only the current probability of a storm but how that probability develops over the life of the storm. Collaborating with neighboring CWAs contributed to workload. Forecasters were already challenged with the new PHI paradigm, collaborating and handing off objects with neighboring CWAs added additional workload from communication. Other significant contributions were sluggish software response and glitches.
- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner