Recent Advances in The Weather Company's 1 - 15 Day Forecasting Guidance Infrastructure

For over fifteen years, the Weather Company relied on variants of the DICast system to drive its 1 - 15 day forecasting guidance. First developed in the late 1990's at the National Center for Atmospheric Research, the DICast (Dynamically Integrated ForeCast) system is an automated forecast system designed to maximize objective forecast skill by blending many inputs and adjusting the blending through continuous and near real-time comparison against observations. Using a gradient-descent nudged machine learning approach, the DICast system combines inputs to generate optimized forecasts with skill generally superior to that of any single input forecast. Various customized versions of the DICast system have been in use for over a decade at The Weather Company.

About two years ago, a significantly updated version of the DICast system was developed to drive the Forecast on Demand (FOD) system (Neilley et al. 2015). This second generation system, named gDICast, or gridded DICast, employed a global gridded architecture with hourly output timesteps to support the spatial and temporal resolution requirements of FOD. This massive system contained some improvements to the DICast subsystems responsible for deriving consumer forecast parameters from input numerical weather prediction output, while preserving much of the underlying DICast framework and technology. The gDICast system represented an advancement over the legacy DICast system, but was mainly a technological effort, rather than a framework to improve the underlying science and meteorological efficacy of the system (Koval et al. 2015).

In fact, key aspects of the gDICast system presented challenges that limited our options for further improvements. The gridded nature of the gDicast system required the storage of a massive, expensive archive of historical forecast and observation data. Further, the large footprint of the gDicast system made it prohibitive to manage a development and testing environment suitable for improving the science of the system, particularly the introduction of more sophisticated self-learning optimization.

Recently, a new 1 - 15 day forecast guidance system was designed and is currently being implemented at TWC. The new architecture marks a clear departure from the long-standing DICast framework. The centerpiece of the new system is a regularized linear regression technique that replaces the gradient-descent nudged approach of the legacy systems (Williams et al. 2016). This new machine learning approach will be applied at approximately 25,000 observation sites, rather than on a massive synthetic grid of over 35,000,000 points. The architecture of this new system is streamlined to be more extensible and manageable for efficiency in developing, testing and deploying improvements. Finally, the new system includes improvements to the forecasting of precipitation variables using machine learning techniques.

This presentation will provide a description of this third generation forecasting system and comparisons with prior versions. Examples will be presented, and we will share some insights on future directions.