A Study of Derived Convective Parameters from Deep Learning Weather Prediction Models

Machine learning (ML) has been used for a number of weather forecasting-related applications, including emulating output of numerical weather prediction (NWP) models. Several of these deep learning weather prediction models are presently open source, allowing for forecasts to be run in real-time and retrospectively by the Cooperative Institute for Research in the Atmosphere (CIRA) in collaboration with NOAA’s Global Systems Lab (GSL). In previous literature, these models have shown promising performance in evaluations of some primary kinematic and thermodynamic fields. However, the relatively coarse vertical resolution of many of these ML-based systems motivates the question as to whether they can capture more complex atmospheric parameters, as these fields tend to be crucial in forecasting specific types of impactful weather, such as flash flooding, hail, and tornadoes.

In this research, we compute and evaluate a variety of derived atmospheric parameters over CONUS that are relevant to convection (e.g., CAPE, shear, precipitable water) using our archive of forecast output from multiple ML-based weather prediction systems. The three industry-led models investigated in this work are GraphCast (Google), Pangu-Weather (Huawei Cloud) and FourCastNet v2-small (NVIDIA): all of which are deep learning models trained on ERA5 reanalysis. The convective parameters derived from each of these model forecasts are evaluated in aggregate across varying forecast lead times (short- to medium-range), and they are compared to ERA5 reanalysis over a two-year period (November 2021 to October 2023). Only select vertical levels (13, mirroring those that are used in the ML model output being evaluated) are used to compute the ERA5 convective parameters in an effort to draw more equivalent comparisons between the models and the reanalysis. We also compare the ML-based forecasts to output from the Global Forecast System (GFS) for a select number of severe weather cases to investigate differences between the parameters derived from the ML systems and those derived by a NWP-based system. All vertical levels are retained in the GFS fields in order to retain characteristics of the output that a forecaster would encounter in an operational forecasting setting. Preliminary analysis suggests that when compared to ERA-5, the derived convective parameters computed from the ML forecasts tend to underpredict precipitable water (PWAT), particularly over the Intermountain West, and they slightly overpredict PWAT over parts of New England and the Southeast. Differences in surface-based CAPE in the ML forecasts and ERA-5 are relatively small over CONUS in the cool season, yet there are substantial discrepancies in the warm season, including overprediction by the ML systems of surface-based CAPE over the Great Plains and underprediction in the Deep South, Ohio Valley, and Northeast.