Using Deep Learning to Predict Error Growth in Model Forecasts

Global weather models observe occasional significant drops in the accuracy of their forecasts often referred to as “forecast busts” in the literature (e.g., Rodwell et al. 2013). Despite recent efforts (Lillo and Parsons 2017; Magnussen 2017; Grams et al. 2018; Magnussen et al. 2019), understanding the causes of these forecast failures remains challenging, but essential. Ensemble spread is often used as a measure of forecast reliability with large spread expected to be associated with poor forecasts. However, Rodwell et al. (2018) argues that ensemble models are often overconfident in these forecast failure. Therefore new techniques would be helpful to predict the occurrence of forecast busts and guide forecast centers to paths for model improvement.

Deep learning techniques are being applied to an increasing number of applications in meteorology such as: prediction of fronts (Kunkel et al. 2018, Lagerquist et al. 2019), severe weather (McGovern et al. 2019, Gagne II et al. 2019) and forecasting geopotential heights (Weyn et al. 2018). Recently deep learning has been applied to predicting model forecast accuracy. For example, Scher and Messori (2018) trained a neural network to predict root mean square error (RMSE) for a 3-6 day forecast using a single time, they found some skill in predicting reduced accuracy. However, this approach did not outperform the ensemble spread.

Investigations of the dynamics of error growth suggests tracking error growth in the first two days is critical since it appears to follow a power-law growth rate in this timeframe followed by a slowdown (Baumgart et al. 2018). This work explores using neural networks to predict model forecast accuracy by adding a temporal component to the neural network that tracks error growth at each forecast time step and uses visualization techniques that provide information on which atmospheric flow regimes are resulting in forecast busts. Preliminary efforts that have concentrated on the detection of forecast busts over Europe using the ECMWF model have shown some promise both in detecting these busts and the flow characteristics associated with these events. Current work is focusing on comparing our results against the ensembles and exploring different deep learning techniques and domains. Given that bust events that meet Rodwell’s criteria are relatively rare, we are also investigating predicting the general error characteristics in these global medium range forecasts.