24AI Evaluation of Ensemble ML Rainfall and Snowfall Forecast Products for HMT FFaIR

During the 2024 Hydrometeorology Testbed (HMT) Flash Flood and Intense Rainfall (FFaIR) experiment and Winter Weather Experiment (WWE), scientists at the Center for Analysis and Prediction of Storms (CAPS) generated probabilistic forecasts of precipitation (rainfall during FFaIR and snowfall during WWE) using machine learning (ML) ensemble. The ML ensemble was trained on and used inputs from a 12-member combined NWP ensemble consisting of 4 FV3-LAM forecasts run in real-time by CAPS and 8 members of the operational HREFv3 ensemble (using 00 UTC and time-lagged 12 UTC forecasts from 4 HREFv3 members). From this ensemble of ML forecasts, ensemble probabilistic forecasts of rainfall or snowfall were produced daily for 6-h periods between 0 and 36 hours of forecast time. Both best-estimate ensemble consensus forecasts and probabilistic forecasts of exceeding specific rainfall/snowfall thresholds were produced.

Individual ML forecasts for each member were generated using a U-Net (a deep learning approach using convolutional neural networks which is designed to identify spatial patterns in images). Approximately two dozen input variables were used for each member, including predictions of wind, temperature, and moisture information at different vertical levels, as well as predictions of reflectivity, precipitation, and precipitable water. Derived variables (such as vorticity and moisture convergence) and static fields (such as terrain height) were also included among the input channels. A patchwise approach was used to generate CONUS-wide ML forecasts; the U-Net generates predictions on overlapping 64x64 patches which are then stitched together to form a CONUS-wide prediction. Neighborhood ensemble probability (NEP) and neighborhood maximum ensemble probability (NMEP) approaches using a variety of neighborhood radii and smoothing settings were evaluated in generating ensemble probabilistic ML forecasts.

Ensemble probabilistic ML rainfall forecast results from the 2024 HMT FFaIR and WWE will be presented, discussed, and evaluated using a combination of full-season objective and subjective statistical evaluation, as well as consideration of forecast performance for specific high-impact cases. The performance and forecast characteristics of forecasts generated using different hyperparameters will be examined, and the implications for operational ML forecast product design will be discussed.