How Long of an Observational Record Is Needed for Skillful ML-Based Forecasts of Excessive Rainfall?

Postprocessing numerical weather prediction output has gained considerable popularity in recent years as machine learning (ML) has emerged as an efficient and viable tool in generating explicit forecasts of weather hazards. As an example, the Colorado State University Machine Learning Probabilities (CSU-MLP) prediction system was developed to provide probabilistic first-guess forecasts of excessive rainfall to aid operational forecasters at the Weather Prediction Center (WPC). The CSU-MLP system uses Random Forests (RFs), reforecasts of the Global Ensemble Forecast System (GEFS/R), and multiple observational datasets of excessive rainfall to produce daily probabilistic forecasts extending out to 8 days in support of WPC operations and their experimental products. The system has shown positive skill out to day 6 regardless of which observational dataset is used to train the RF models. Interestingly, the two primary excessive rainfall datasets being used for model training have significantly different lengths – one has nearly 10 years of observations and the other only 3.5 years. The early results from these prediction systems raises the question: how long does the observational record need to be in order to obtain skillful ML-based forecasts? This presentation will detail a number of experiments in which the training lengths are varied in order to assess forecast skill sensitivity. RFs are trained with 1, 2, 3.5, and 10 years of data and the trained models are applied to an unseen 2.5 year testing dataset to assess forecast skill differences. While these experiments are scientifically motivated, they will also provide a foundation for how to operationally implement these types of ML-based systems, and what resources will be needed to retrain ML models when the underlying dynamical models (e.g., GEFS) undergo upgrades.