Evaluating the Performance of AI-based and Traditional NWP Guidance for High Impact Winter Storms over the U.S. East Coast

Over the last couple years, AI-based numerical weather prediction (NWP) has emerged to become a promising tool in weather forecasting. Some models, like the ECMWF AIFS, have been shown to outperform traditional NWP systems for particular metrics and variables. However, questions remain over their reliability in day-to-day use and especially for extreme weather events. Therefore, further analysis is necessary to compare these systems with more traditional NWP methods. This study will analyze the performance of AI-based NWP systems (ex. GFS Graphcast) for high impact weather events with a goal of identifying systematic errors in predictions, as well as areas of improvement or degradation over traditional NWP.

This study will focus on predictions of winter storms over the East Coast of the United States occurring within the last three years. Several key meteorological variables will be examined, including surface pressure, wind speed/direction, surface and upper-air temperature and moisture as well as the quantitative precipitation forecast (QPF). The errors of the AI-based weather systems compared to reanalysis data and observational analysis (ex. RTMA) will be calculated to identify differences in cyclone structure and physical processes. These biases will also be compared to errors made by traditional NWP models to see where each suite excels. By comparing the performance of traditional NWP and AI-based models at predicting winter storm features as a function of forecast lead time, the practical predictability of snowstorms can be re-evaluated. As a result, this work can allow for forecasters and research meteorologists to have higher confidence in utilizing AI-based NWP in a day-to-day operational setting.