Neighborhood and Object-Based Probabilistic Verification of the OU MAP Ensemble from the 2018 Hazardous Weather Testbed Spring Forecasting Experiment

A 10 member ensemble forecast was initialized from the GSI-based hybrid EnsVar data assimilation system at 0000 UTC in real time during the 2018 HWT SFE, with forecasts going out to the 36-hr lead time. Many ensemble systems were contributed to the 2018 HWT in order to answer scientific questions about the ensemble design. However, the conclusions that can be drawn from comparative systematic verification studies can at times depend on the choice of verification metrics, especially when using convection-permitting models to predict severe weather and convective precipitation systems. The focus of this study is on the systematic application of a new object-based probabilistic verification technique in comparison to a more commonly applied neighborhood technique, in the context of severe weather forecasting in the HWT.

It will be shown that the relative performance of the OU MAP ensemble probability forecasts at 21-27-hr lead times for all cases during the 2018 HWT is different in many instances when quantified with the object-based technique rather than the neighborhood technique. These differences result from the different conceptual emphases of the different techniques, both of which are consistent with different aspects of subjective evaluation. In particular, while the neighborhood method focuses primarily on the approximate locations of convection, the object-based method focuses on aspects like storm mode and upscale organization that are also important for severe weather forecasters. The multi-variable object attributes are also shown to provide conditional severe weather probabilities, which are not easily verified with the neighborhood technique. Overall, it is shown that a more comprehensive picture of the performance of the 2018 OU MAP ensemble probability forecasts can be obtained by considering both the neighborhood and object-based verifications together than by either of them by itself.

The presentation will also include a summary of feedback on the real-time application of object-based ensemble visualization from the 2018 HWT, and future plans for the 2019 HWT, as a part of the effort to transition object-based probabilistic ensemble post-processing and verification into operations.