An Instantaneous Self-Assessment of Forecaster Cognitive Workload

Since 2010, the Phased Array Radar Innovative Sensing Experiment (PARISE) has been exploring the impacts of higher-temporal resolution radar data on NWS forecasters’ warning decision processes during simulated warning operations. Performance and retrospective recall data show that 1-min PAR updates are beneficial to forecasters in both severe hail/wind and tornado warning scenarios. However, concern of possible data overload has been expressed within the meteorological community. The most recent 2015 PARISE therefore assessed forecasters’ perceived cognitive workload. In this experiment, 30 NWS forecasters worked nine archived phased array radar cases in simulated real time independently. These cases presented a variety of storm modes and weather threats (i.e., null, severe hail/wind, and tornado), and forecasters had the opportunity to work with approximately 1-min, 2-min, and 5-min phased array radar volumetric updates. 

The Instantaneous Self-Assessment (ISA) tool was used during a retrospective recall of worked events to collect forecasters’ subjective ratings of their cognitive workload. These ratings were collected for every 5-min interval in each case. On a scale of 1–5, forecasters expressed the level of their mental effort as: under-utilized (1), relaxed (2), comfortable (3), high (4), and excessive (5). These ratings also relate to how much spare mental capacity forecasters felt they had: very much (1), ample (2), some (3), very little (4), and none (5). For each case, forecasters’ median ISA ratings were at a level 2 or a level 3 regardless of temporal resolution. This result suggests that overall forecasters were not cognitively overloaded.

Despite similarity in the median workload results, the distributions of ISA ratings for forecasters’ with more rapidly-updating phased array radar data were skewed towards higher levels. Most notably, participants using 1-min phased array radar updates during the tornado cases experienced higher levels (4 and 5) of cognitive workload more frequently than forecasters using the more slowly updating phased array radar data. Given that these higher levels of cognitive workload are of most concern, forecasters’ associated reasoning for all level 4 and level 5 ISA ratings were analyzed (N=183). Six categories that described the reasoning were identified as: 1) Storm Characteristics, 2) Warning, 3) Case Startup, 4) Temporal Resolution, 5) Technical Frustrations, and 6) Personal. The prevalence of each category was assessed first as a function of phased array radar update speed and second as a function of weather threat type. Descriptions of these categories and the results of their importance will be shared in this presentation.

Forecasters’ workload profiles were also analyzed for each case. The 5-min ISA rating patterns tended to be either flat (i.e., little or no change in workload) or fluctuating (i.e., multiple increases and decreases in workload). We did not observe a relationship between the type of workload profile pattern and temporal resolution/case type. We did, however, find that 21 of the 30 participating forecasters consistently reported either a flat or a fluctuating workload profile in most of their worked cases. This workload behavior tendency suggests that personality traits may have also been an influential factor in forecasters’ perceived cognitive workload. Findings from other studies looking at the relationship between personality and cognitive workload support this suggestion.