Evaluation of Tropical Cyclones in Global Data-Driven Forecasting Models

While machine learning has experienced a steep upswing during the last decade, its value for global weather prediction modeling has only been proven in the last two years. Accelerated by the involvement of various tech companies and their provision of appropriate computing resources, a few data-driven models have been released, showing comparable or better skill than ECMWF's IFS model for various parameters. In the corresponding papers, forecast improvements are often demonstrated by subjectively selected tropical cyclones (TCs), and comparisons of forecast errors are often based on TC trackers, which are applied to the models in different versions.

This contribution presents a systematic evaluation of the representation of TCs in the current generation of global data-driven forecasting models. Leading models, including FourCastNet, Pangu-Weather, GraphCast, and FuXi, all trained on predominantly or entirely on ERA5, are run starting from ECMWF’s operational initial conditions. ECMWF’s TC tracker is then consistently applied to all models and the predicted TCs are verified against the IBTrACS dataset. Evaluation results show that TC prediction can still be advanced in areas where it has been speculated that predictability limits had been reached. On the other hand, the findings also shed light on deficiencies in the data-driven models that are due to resolution and limitations in the training data.