Severe Hail Forecasting Evaluation: Machine Learning and Severe Weather Proxy Variables

Convection-Allowing Model (CAM) ensembles are transitioning from research to operational use for forecasting the location, likelihood, and intensity of severe weather events 1-2 days before they occur. Proxy variables for severe weather, such as updraft helicity, have been used to anticipate the timing, severity, and mode of severe thunderstorms for many events, but these proxy variables have inherent limitations. They do not directly predict the presence of severe weather hazards such as tornadoes and hail, and the neighborhood probabilities derived from them are not calibrated to the observed rate of occurrence of severe weather hazards, often leading to overconfident forecasts containing large false alarm areas. Statistical and machine learning techniques can provide a link between model proxy and environmental variables and observations while also estimating the uncertainty associated with the forecasts. This presentation demonstrates the value added by using probabilistic forecasts of severe hail calibrated using machine-learning methods as opposed to forecasts of proxy variables and explicit hail forecasts using physics-based methods.

Hail forecasts are derived from forecast ensembles produced by the 2015 Center for Analysis and Prediction of Storms (CAPS) Storm-Scale Ensemble Forecast system and the 2015 NCAR CAM Ensemble. Radar-indicated hail observations are obtained from the NOAA Multi-Radar Multi-Sensor (MRMS) radar mosaic Maximum Estimated Size of Hail (MESH) product. A tracked object approach is used to identify potential hailstorms in the forecast and observed grids. The enhanced watershed technique is used to identify storms, and storm-motion-corrected centroid distance is used to track storms. A weighted average of centroid, time, and duration differences is used to match predicted and observed storms. Three machine-learning learning methods are evaluated: random forest, gradient boosted regression, and multinomial logistic regression. Models are trained to predict probability density functions of hail size as well as temporal and spatial corrections to the tracks. Neighborhood ensemble probabilities are derived from the machine learning methods as well as from proxy variables and physics-based hail algorithms. The different methods are evaluated using traditional verification statistics, and select events are highlighted. Preliminary results show that the machine learning methods produce improved forecasts compared to the other approaches though the choice of machine learning algorithm has only a marginal impact.