Comparison of Object-Based and Grid-Based Verification of Warn-on-Forecast System HAILCAST Forecasts

Hail is a significant convective hazard in the United States, causing damage to property and agriculture of over $5 billion annually. Forecasts of hail size and occurrence are becoming increasingly more detailed in an attempt to mitigate this hazard, yet fully determining the accuracy of these forecasts remains a difficult problem. For example, all high-resolution convective forecasts suffer from a “double penalty”; a small error in time or space can cause penalties for both failing to detect and a false prediction. Convective hazard forecasts, such as hail, have an additional “unfair penalty” problem: a convection-allowing model (CAM) must first generate convection in the right location and time, and then the convective hazard must be correctly predicted. A variety of object-based verification configurations have been designed by the investigators to alleviate these issues. Two of the configurations are implementations of the Model Evaluation Toolkit (MET) Method for Object-Based Diagnostic Evaluation (MODE) with configurations designed to match storm objects on both cellular scales and mesoscales. The third configuration is a simplified object matching scheme based on MODE and designed for application to short-term, storm-scale forecasts.

All three of these object-based verification methods, in addition to traditional neighborhood grid-based verification, are used to validate hail size forecasts from the National Severe Storms Laboratory (NSSL) Experimental Warn-on-Forecast System (WoFS). The WoFS is a 18-member ensemble forecast system designed to provide rapidly-updating, probabilistic guidance of severe hazards, including hail. As part of the 2019 NOAA Hazardous Weather Testbed (HWT), the WRF-HAILCAST hail forecasting module was incorporated into the WoFS ensemble to produce hail size forecasts at 5-minute intervals every 30 minutes, out to 6 hours. Results from the grid-based and three object-based hail forecast verification methods are compared.