Weather Forecasting - What Have We Learned and Where We Are Headed

Numerical weather prediction (NWP) has evolved significantly since The Weather Company (TWC), an IBM Business began running NWP systems over 20 years ago. It is interesting to look back on the state of technology 20 years ago. At that time processor speed was measured in MHz, storage space was generally available in MB, and NVIDIA released the first GPU enabled graphics card.

Around that same time, TWC was working on its first NWP system, which had the trade name of “Predictor”. The Predictor model, based on the MM5, was run at the customer’s location, with a custom domain covering a metro region at 12km resolution. This gave the forecaster and viewer audience a view of the weather in the immediate vicinity, but lacked the larger scale view. The Predictor model was replaced by a WRF based model called “RPM” in the late 2000’s, and was run on a national scale with a domain covering the entire United States at 12km resolution. Over the years as computing power, storage space and network speeds have all increased, the model resolution, scale and output frequency have continued to increase. Today, the RPM is available at up to a 4km resolution with 30 minute output frequency over the US and Europe and 13km world-wide.

Additionally, in 2013 TWC launched the Forecast on Demand (FOD) service, bringing many forecast-generating aspects into a real-time on-demand process. This system is comprised of 178 different models and updates the forecast information for 2.2 billion locations on the planet every 15 minutes. FOD provides forecasts out to 10 days, with some additional seasonal outlooks for longer-term weather insights.

In 2016, TWC was acquired by IBM. Given IBM’s history with supercomputing and big data, our scientists were extremely excited about the potential to take numerical weather prediction to another level. This vision is starting to take shape, as TWC is nearing the release of a global weather prediction system that will run at convective-allowing scales over large parts of the world. The system, referred to as IBM GRAF (Global High-Resolution Atmospheric Forecast System), is the result of a collaboration including TWC, IBM Research and NCAR. This new forecast model will replace the older WRF based technology used in the RPM. It will be driven by MPAS (Model for Prediction Across Scale) and will incorporate Gridpoint Statistical Interpolation (GSI) software for the data assimilation.

The data assimilation represents a very unique aspect of the GRAF model. The new system will pull from previously untapped data sources to give the model a better picture of the state of the atmosphere when the model initializes. Outside of traditional initialization sources such as METAR and SYNOPs observations, radiosondes, satellite and radar, the GRAF model will also incorporate wind speed and pressure data from aircraft, as well as pressure sensor data from cell phones (where users have opted in to sharing that information). This makes GRAF the first model to take advantage of data from the Internet of Things (IoT) to produce a more accurate representation of the initial conditions for the model.

To handle this massive amount of data, and generate model output at a high resolution (3km) for much of the world, GRAF will run on an IBM supercomputer driven by Power9 CPUs and nVidia GPUs. The supercomputer will contain hundreds of CPUs and nearly 300 nVidia GPUs. The IBM Research team worked closely with NCAR to develop a version of MPAS that runs nearly 5x faster on GPUs than MPAS runs on CPUs. This performance boost enables TWC to run GRAF globally and to cover 35% of the world at 3km resolution. This means that the IBM GRAF model will be the only global model with the power to resolve thunderstorm level detail over most land masses. The end result of everything described above are two global models, one generating a 15-hour forecast every hour, and another updating a 72-hour forecast every 6 hours. By generating the data at a global level, we have eliminated the uncertainties that fixed domain boundaries can introduce. The model’s resolution will be variable. For the short-term forecast, it will be run at 3km over most of the land and 13km-15km elsewhere. The long-term forecast will be 4km over North America and Europe, and 13km-15km elsewhere.

Forecasting using Numerical Weather Prediction has come a long way in the last twenty years. At TWC we are looking forward to continuing to push the boundaries of technology as we strive to create the most accurate global forecasts.