Evaluation of a Machine Learning Postprocessor to Mitigate QPF Errors for Mesoscale Convective Systems

Convection-allowing grid spacing models have been shown to improve simulation of heavy rain with warm-season convection. Although these models improve the skill in quantitative precipitation forecasts (QPF), errors in precipitation intensity and location remain problematic, especially in critical scenarios such as flood prediction. Our research aims to reduce these errors through the use of machine learning tools. Using the Method for
Object-based Diagnostic Evaluation (MODE), we have characterized the errors within the High-Resolution Ensemble Forecast (HREF) members for approximately 300 mesoscale convective events occurring in the warm seasons from 2018-2023. Additionally, we gathered information on atmospheric parameters from the Storm Prediction Center mesoanalysis for each event. Through the assimilation of this data, we created a machine learning (ML)-based postprocessor that corrects for displacement errors within the HREF ensemble. This ML-based postprocessor was run in a pseudo-operational mode in real-time as part of the 2024 Flash Flood and Intense Rainfall (FFaIR) experiment. Probability-matched mean (PMM) and probability of exceedance graphics were generated and evaluated subjectively and objectively.

The average precipitation centroid displacement error for the post-processed QPF within our test set was 103 km, which represents an improvement compared to any individual HREF member and a 19 km improvement when compared to the best member, the NSSL 12z. Critical success index (CSI) scores throughout FFAIR showed that the post-processed QPF performed about the same as or better than the HREF ensemble at thresholds above 0.5 inches for the 2024 warm season. This was supported by the subjective evaluation, where FFAIR participants expressed that our tool seemed to be especially useful for higher thresholds, and 69% of participants felt that our PMM guidance performed the same as or better than the HREF PMM guidance.

We will present the ML tool development, the subjective and objective evaluations of our tool at FFaIR through CSI and Equitable Threat Scores, as well as additional objective verification results. In future work, we intend to enhance the capabilities of this tool by correcting for intensity errors within the HREF ensemble, and may discuss preliminary results from the development of that ML tool if available at the time of the conference. The anticipated outcomes of this research include the development of multiple forecast products, such as probability-matched means and augmented probability-quantile precipitation forecasts (PQPFs).