Pushing the Atmospheric Predictability Limit: Nonlinear Sensitivity Analysis for Optimized Initial Conditions

Analysis of operational forecasts suggests that the atmospheric predictability limit for the mid-latitudes is currently around 10 days (Zhang et al., 2019) and may be improved to 16-23 days using ensemble prediction (Buizza & Leutbecher, 2015). Recently, deep-learning weather models have shown breakthrough performance in forecast skill relative to operational models. Moreover, the frameworks of these models provide the tools needed for fully nonlinear sensitivity analysis, allowing for novel approaches to estimating atmospheric predictability. The traditional method for estimating weather forecast sensitivity to initial conditions uses tangent linear and adjoint models, which are limited to short lead times due to linearization around a control forecast. Automatic differentiation in machine-learning frameworks enables deep searches for optimal initial conditions that are not practical using adjoint methods. Its key innovation is the ability to quickly compute gradients of the output with respect to the inputs, enabling the iterative adjustment of initial conditions through gradient descent to minimize forecast error.

Using the GraphCast model and perturbing ERA5, we demonstrate the power of this method for a large sample series of 10-day global forecasts, achieving a 70-80% reduction in forecast error over a control forecast from ERA5 initial conditions. For longer lead times, preliminary results show the optimized initial conditions reduce 20-day forecast errors by 30-50%. The method remains effective for forecasts out to about 25 days, showing improvements of 10-15%, and suggesting that the atmospheric predictability limit is at least this long. Importantly, preliminary results show that error reduction is consistent across a diverse set of initialization times, suggesting that these improvements are not isolated to particular atmospheric states. Finally, the perturbations introduced by this approach are generally of the same order as analysis error, indicating that they are likely resolvable by operational forecasting systems.

Buizza, R., & Leutbecher, M. (2015). The forecast skill horizon. Quarterly Journal of the Royal Meteorological Society, 141 (693), 3366–3382.

Zhang, F., Sun, Y. Q., Magnusson, L., Buizza, R., Lin, S.-J., Chen, J.-H., & Emanuel, K. (2019). What is the predictability limit of midlatitude weather? Journal of the Atmospheric Sciences, 76 (4), 1077–1091.