Impacts of Rapidly-Updating PAR Data on the Forecaster Warning Decision Process: A Qualitative Approach

The Phased Array Radar Innovative Sensing Experiment (PARISE) recognizes the importance of collecting qualitative information regarding forecasters' warning decision processes. Although performance measures such as lead time and verification provide some insight into how rapidly-updating PAR data might impact forecaster performance, a deeper understanding of what these impacts are and why they exist can only be made possible through the analysis of qualitative data. For this reason, the 2013 PARISE allocated a large portion of data collection time to obtaining such information.

The goal of the 2013 PARISE was to investigate how rapidly-updating PAR data impacts forecasters' warning decision processes during severe hail and wind events. To accomplish this goal, twelve NWS forecasters were recruited and assigned to either a control (5-min PAR updates) or experimental (1-min PAR updates) group. Forecasters worked two archived PAR cases in simulated real time. The first case presented a marginally severe hail event, and the second case presented a downburst event associated with severe hail and wind. Forecasters were asked to work the case and make warning decisions according to their normal routines. All PAR data were displayed in the Advanced Weather Interactive Processing System-2 (AWIPS-2), where base velocity, reflectivity, and spectrum width could be viewed. Upon completion of the case, participants worked independently with a researcher to complete the recent case walkthrough (RCW). The RCW is a form of cognitive task analysis that was also employed in the 2012 PARISE. Participants are required to “sweep” through the case three times. In the first sweep, participants watch a playback video of their onscreen activity during the case; they are asked to recall everything they were seeing, thinking, and doing at that time, and all verbalizations are recorded in a timeline by the accompanying researcher. In the second sweep participants review the timeline and make corrections or additions as they see necessary. Lastly, the third sweep requires participants to identify their key decision points (i.e., warning decisions), and they are asked a set of probing questions designed by the researcher.

The timelines created from the RCW provide a wealth of rich, detailed descriptions of forecasters' warning decision processes. The analysis of these data was completed in two parts. First, the data were thematically coded according to a situational awareness framework. Codes including perception, comprehension, and projection were counted for each participant for both cases. Results show that the experimental group perceived significantly more information in the radar data during both cases than the control group (case 1 p=0.045 and case 2 p=0.041). We hypothesize that the significantly larger number of perceptions made by the experimental group resulted in improved comprehension, projections, and overall decisions. The second part of the analysis identified contextual evidence within the data of how 1-min PAR data proved to be of value to the experimental group by comparing their warning decision processes to that of the control group's. In addition to the experimental group demonstrating an ability to observe radar precursor signatures earlier than the control group, key examples show that the experimental group were able to 1) more accurately project storm motion, 2) correctly identify what weather threat storms posed more often, and 3) make more informed tornado warning decisions.