Role of Automated Guidance when Forecasters Issuing Probabilistic Hazard Information (PHI) during 2016 FACETs PHI-Prototype Hazardous Weather Testbeds

Role of Automated Guidance when Forecasters Issuing Probabilistic Hazard Information (PHI)

during 2016 FACETs PHI-Prototype Hazardous Weather Testbeds

Chen Ling¹, Joseph James¹, Christopher Karstens^2,3, Kristin Calhoun^2,3, James Correia^2,4,

Alan Gerard³, Lans Rothfusz³

¹Department of Mechanical Engineering, University of Akron, Akron, OH

²Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,

Norman, OK

³NOAA National Severe Storms Laboratory, Norman, OK

⁴NOAA Storm Prediction Center, Norman, OK

In 2016 FACETs PHI-prototype Hazardous Weather Testbed (HWT), forecasters issued probabilistic hazard information (PHI) based on three types of automated guidance- probability of severe thunderstorm (ProbSevere), tornado, and lightning. Forecasters created and managed probabilistic hazard information (PHI) objects based on their understandings of the severe weather development, with the help of automated guidance. Based on field observations during HWT, survey and interview results with forecasters, this study summarizes the role of automated guidance as forecasters issue Probabilistic hazard information (PHI), and provides recommendations on improve the design of the automated guidance to better assist forecaster’s task.

In general, the forecasters used the automated guidance as the “first guess” information in helping them to prioritize areas to focus their attentions on. When situations get busy, they chose to focus on the most severe threats and re-calibrate the information provided by the automated guidance, while leaving the automated guidance to provide information on the less-severe threats. They liked the availability of automated guidance for lightning. 

The forecasters noted issues with the current automated guidance design. For ProbSevere and lightning objects, forecasters expressed concerns when the automation merged or split an object they had previously edited and as a result often an object ID or previously edited information was dropped, this resulted confusion and frustration. When the automated object behaves differently than what they expected, the forecasters had to spend extra efforts in monitoring them and deciding whether to “take over.” This “lack of control” over the automated objects behavior created extra workload for the forecasters.

In terms of confidence in the automated guidance information, forecasters stated they were neutral to somewhat-confident in the automated guidance. Forecasters indicated that they used the guidance to validate their interrogation when it agreed with their understanding of the meteorology of the event. However if the guidance disagreed with their understanding, they were more trusting of their own analysis based on their years of experience. Due primarily to a lack of experience and training with lightning data and forecasting, the forecasters tended to trust and rely on the guidance and associated probabilities more frequently.  They expressed concerns that less-experienced forecasters may place too much trust in the automated guidance information, without enough weather interrogation. As the algorithm and design of the automated guidance being improved based on feedbacks from this HWT, we expect to see higher level of forecaster trust towards guidance information.