Neighborhood Verification of 1-h Probabilistic Low-Level Rotation Forecasts from the NSSL Experimental Warn-on-Forecast System.

Although the skill of the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e) mesocyclone guidance has been demonstrated in a deterministic framework, extension to probabilistic framework is warranted. Since tornadoes are unresolvable in current operational models, three model surrogates for mesocyclones are used. These include low-level (0-2 km Above Ground Level [AGL]) updraft helicity, midlevel (2-5 km AGL) updraft helicity, and low-level vertical vorticity. Given that traditional forecast performance metrics often score poorly due to the difficultly of exactly matching forecasts and observations on convections-permitting grids, a double-neighborhood verification approach is used. First, forecast probabilities are upscaled using the neighborhood maximum ensemble probability (NMEP; Schwartz and Sobash 2017 ) method. Second, a forecast verification neighborhood (FVN) is used to score the NMEPs. The combination of the two neighborhoods helps prevent operationally tolerable forecast location errors from unduly decreasing forecast probabilities and skill/reliability.

The NEWS-e forecasts were generated during the 2017-2018 NOAA Hazardous Weather Testbed Spring Forecasting Experiments and are verified against low-level azimuthal shear rotation tracks from the NSSL Multi-Radar/Multi-Sensor (MRMS) product suite. Given the rarity of observed rotation tracks, forecast NMEPs are evaluated using the performance diagram, which summarizes many traditional forecast metrics while neglecting “correct negatives” (i.e., correct forecasts of non-events). Reliability of the NMEPs within a neighborhood is also calculated. Forecast probabilities valid over lead times of 0-60 min are generated using NMEP neighborhoods of 9, 15, and 30 km and verified using FVN of 9, 15, 30 km. The best results are obtained using a 30-km NMEP neighborhood and 15-km FVN, for which the maximum CSI is 0.35 with corresponding POD and Success Ratio of 0.8 and 0.45, respectively. For the same NMEP neighborhood and FVN, forecast probabilities of 0 – 50% are fairly reliable while higher probabilities are slightly over-forecasted, but still skillful.