A Convection-Permitting Reforecast for Training High-Resolution Deep Learning Numerical Weather Prediction

Deep learning numerical weather prediction (DLNWP) algorithms require high-quality training data sets with sufficiently large numbers of samples in order to train and validate models that can predict the expected evolution of a forecast and its uncertainty. For DLNWP at convection-permitting resolution, the most ideal data set would be a reanalysis with high spatial and temporal resolution, perhaps 15 min time steps, and with realistic, spatially coherent details of thunderstorm-scale variability. While there are some products such as NCAR’s CONUS404 data set with some of these characteristics, more training data are welcomed. In lieu of a much more complicated reanalysis, The Weather Company (TWC) and NVIDIA have collaborated on the development of a reforecast data set using TWC’s version of the global NCAR MPAS model, with mesh refinement over North America and Europe to 4 km grid spacing and 15 km elsewhere. The stored model output fields are reasonably comprehensive, including the storage of forecast variables on 50 vertical levels, including cloud hydrometeors, and most common surface and land variables. Forecasts are conducted to +27 hours lead time, to provide a 3-hour spin up followed by a complete diurnal cycle. 1825 reforecast cases were conducted from 2003 to 2024 initialized from ERA5 reanalyses, with initial condition case dates selected preceding US landfalling tropical cyclones, severe local storms, winter weather, and heavy precipitation events across major contiguous US river basins. Data are stored in chunked zarr format.

The reforecast data will be made freely available to enterprise participants through an Amazon S3 bucket, though other commercial users are required not to use the data for their own profit making purposes for a period of one year.

Other talks at AMS Annual may discuss results of DLNWP systems developed using this reforecast data set.