Results and Verification for Machine-Learning-Based HREFv2 and HRRRE Hail Forecasts from the Spring and Summer of 2019

During the spring and summer of 2019, researchers from the Center for Analysis and Prediction of Storms (CAPS) and the National Center for Atmospheric Research (NCAR) produced 12-36 h forecasts of probability of severe hail, probability of significant-severe hail, and maximum hail size, using random forest machine learning (ML) methods, for version 2 of the High Resolution Ensemble Forecast (HREFv2) and the High Resolution Rapid Refresh Ensemble (HRRRE). These forecasts were produced daily in real-time and, during the 2019 Hazardous Weather Testbed (HWT) spring forecast experiment (SFE), were provided to participants for evaluation and use in the generation of experimental forecasts.

To produce forecasts of hail, ML models are trained for HREFv2 and HRRRE using Maximum Estimated Size of Hail from the WSR-88D radar network as an observed hail proxy for training. As input, the ML models use a set of 2-dimensional variable fields from the specified ensemble forecast (HREFv2 or HRRRE), including both storm variables (those directly related to convective storms; e.g. convective available potential energy (CAPE), column maximum reflectivity) and environmental variables (those representative of the larger-scale environment; e.g. temperature and winds at different vertical levels). Using column-maximum updraft speed, potential storm objects are identified in the model data. A random forest model predicts whether a given storm object will produce hail, and, if it is predicted to produce hail, a second random forest model predicts the size distribution of that hail via a gamma distribution. The predicted hail size distributions are then used to generate probabilistic hail forecasts. Finally, the hail forecasts are calibrated using isotonic regression to improve the reliability and bias performance of the forecast and generate a range of probabilities closer to that of human-generated severe hail outlooks.

Overall, during the spring and summer of 2019, these ML-based hail forecasts exhibited good skill in predicting severe hail using both HREFv2 and HRRRE data. Calibration via isotonic regression using local storm reports as the target dataset yielded forecasts with excellent reliability; the objective skill of the ML-based forecasts in terms of equitable threat score and other skill metrics is comparable to or better than that of severe hail forecasts produced using common severe weather proxy variables (such as updraft helicity). Detailed verification will be presented along with results from individual cases and subjective evaluations from 2019 HWT SFE participants.