Object-Based Verification to Inform Design and Real-Time Use of the Warn-on-Forecast System (WoFS)

The Warn-on-Forecast System (WoFS) is designed to provide probabilistic guidance of convective storm hazards between the time scales of National Weather Service watch and warning products. WoFS initial conditions are produced through rapidly-cycled (15-min) assimilation of Doppler radar and satellite observations, which enables realistic analyses of ongoing thunderstorms. These analyses can result in accurate short-term prediction of the track, evolution, and severity of individual convective storms. This focus on short-term prediction of discrete events makes object-based verification methods an attractive option for evaluating the quality of WoFS guidance and object-based techniques have been extensively applied to WoFS forecasts using radar and satellite-based proxies for thunderstorms.

This study will present results using simple methods to identify and match composite reflectivity objects in WoFS forecasts and Multi-Radar Multi-Sensor (MRMS) observations to assess the quality of WoFS thunderstorm forecasts. A dataset of 174 WoFS cases between 2017 and 2021 is used to evaluate two aspects of WoFS thunderstorm forecast quality: 1) WoFS accuracy relative to storm age from convection initiation (CI), and 2) WoFS accuracy relative to storm size. It is found that the accuracy of WoFS thunderstorm guidance is strongly correlated with storm age relative to CI, with mean probability of detection at three hours of lead time increasing from ~0.45 pre-CI to 0.7 when storms have been ongoing for more than one hour prior to WoFS initialization. The sensitivity of WoFS accuracy to storm age will be extended through evaluation of sensitivity to object size. It is hypothesized that a sharp decline in skill will be found as storm size decreases. Quantifying the variation of WoFS accuracy with storm age and size informs future system design choices and allows WoFS end users to identify storms that are more or less likely to be accurately predicted by the system, resulting in more effective real-time use of WoFS.