Probabilistic Calibration for ML-Based Extreme Event Prediction

Session/Conference: The 24th Conference on Artificial Intelligence for Environmental Science

Title: Probabilistic Calibration for ML-Based Extreme Event Prediction

Authors: Mariah Pope, Maxfield Green, Ashley Payne, Stylianos Flampouris, Arun Chawla

Severe weather events are meteorological hazards that can cause substantial damage and injury, but weather models often struggle to predict the location and true probability of such events. Here, we introduce a modeling and probability calibration framework tailored to improve the accuracy of rare event probabilistic forecasts. The application of the proposed methodology is focused on the prediction of rare or severe weather phenomena such as severe thunderstorms and tornadoes. Traditional scoring metrics and calibration techniques fall short in the context of infrequent weather event forecasting. They can fail to adequately address data imbalances and probability calibration challenges inherent in rare event prediction. This necessitates the development and use of calibration methods tailored to infrequent events.

We present a methodology to first model severe weather, such as hail, with historical storm reports paired with atmospheric data from gridded sources such as ERA5 and HRRR. We then calibrate the probabilistic predictions to reflect true event probability. Similar to Taillardat et al. (2023), the proposed calibration techniques involve the optimization of sensitive skill metrics via a scaling parameter and employing decision thresholds to optimize classifier validation metrics. Through case studies on hail climatology estimation and severe thunderstorm forecasting, the framework demonstrates forecast reliability and sharpness improvements, offering a high-resolution, data-driven addition to Storm Prediction Center’s severe weather outlooks.