Session 4A Pure AI and Data-Driven Weather Forecasts II: Highlights on AI Forecast Performance and Evaluation

Monday, 29 January 2024: 4:30 PM-6:00 PM
345/346 (The Baltimore Convention Center)
Host: 23rd Conference on Artificial Intelligence for Environmental Science
Submitters:
Daniel Rothenberg; Stephan Rasp; Stephan Hoyer, Google, Mountain View, CA and Waylon G. Collins, National Weather Service, Corpus Christi, TX
Cochairs:
Christina E. Kumler, CIRA, Boulder, CO and Daniel Rothenberg

Recent advances in AI modeling techniques - particularly deep neural network architectures and infrastructure suitable for training on massive-scale datasets – have brought a renewed interest in developing “pure” AI-driven weather forecasts. Such a forecast would involve assembling some initial representation of the state of the atmosphere, feeding it into an AI model, and producing a rich set of dense, spatiotemporal output fields similar to the output of a numerical weather prediction (NWP) system; some researchers refer to such models as “AI NWP” (e.g. [1]). Unlike traditional NWP, pure AI forecasts are optimized end-to-end for forecast accuracy based on historical data. A proliferation of such models has been published in the early 2020’s, many of which perform favorably using typical large-scale forecast quality metrics relative to state of the art numerical forecasts for precipitation nowcasting [2] and medium-range [3, 4, 5, 6] prediction. 

The rapid pace of development of these models, backed by non-traditional players in the weather enterprise (specifically “big tech” companies which may not have developed proprietary weather forecasts or data products in the past but have significant experience with very large data and AI applications) suggests that a new frontier in weather prediction is rapidly opening up. In this session, we invite perspectives, commentaries, and demonstrations of new models from this new frontier, and encourage thoughts on questions including the following:

  • What are the remaining barriers towards developing – and operationalizing – “pure” AI or “AI NWP” forecast systems? 
  • Given the rapidly changing frontier of AI research and development, are there novel model architectures or problem formulations that we may see emerge as a new paradigm for developing pure AI forecast systems?
  • How will AI NWP products co-exist with the extant and continually evolving catalog of operational NWP systems? 
  • Pure AI systems at different levels of completeness and maturity have been developed for precipitation nowcasting, medium-range weather and even S2S forecasts. Which weather forecasting timescales and applications will be hardest for pure AI approaches?
  • Are traditional skill metrics still adequate for evaluating “non-physical” AI NWP forecasts? If not, how should these forecasts instead be evaluated?

We especially welcome perspectives on how the broader AI and NWP communities might come together to further accelerate this weather forecasting revolution. Prospective authors may consider submitting abstracts that provide a performance assessment of predictions from their data-driven ML models which include comparisons to benchmark predictions from single deterministic NWP models, and/or an ensemble of NWP models. This session will encourage research into the use of purely data-driven ML models to generate predictions with skill comparable to that from state-of-the-art NWP models used by operational meteorologists worldwide.

[1] Chen, Kang, et al. "FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead." arXiv preprint arXiv:2304.02948 (2023).

[2] Espeholt, Lasse, et al. "Deep learning for twelve hour precipitation forecasts." Nature communications 13.1 (2022): 5145.

[3] Pathak, Jaideep, et al. "FourCastNet: A global data-driven high-resolution weather model using adaptive fourier neural operators." arXiv preprint arXiv:2202.11214 (2022).

[4] Lam, Remi, et al. "GraphCast: Learning skillful medium-range global weather forecasting." arXiv preprint arXiv:2212.12794 (2022).

[5] Keisler, Ryan. "Forecasting global weather with graph neural networks." arXiv preprint arXiv:2202.07575 (2022).

[6] Bi, Kaifeng, et al. "Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast." arXiv preprint arXiv:2211.02556 (2022)

Papers:
4:30 PM
4A.1
A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models (Core Science Keynote)
Imme Ebert-Uphoff, CIRA, Fort Collins, CO; and J. Q. Stewart, K. A. Hilburn, J. T. Radford, R. T. DeMaria, R. Chase, R. A. Lagerquist, C. White, Y. Lee, J. Apke, K. D. Musgrave, L. Ver Hoef, C. E. Kumler, M. S. Wandishin, J. Duda, I. Jankov, and D. D. Turner

5:00 PM
4A.2
Evaluation of Tropical Cyclone Track and Intensity Forecasts from Purely ML-based Weather Prediction Models, Illustrated with FourCastNet
Robert T. DeMaria, CIRA, Fort Collins, CO; and M. DeMaria, G. Chirokova, K. Musgrave, J. T. Radford, and I. Ebert-Uphoff

5:15 PM
4A.3
AI-Models: A Tool for Making Forecasts with Data-Driven NWP Models
Baudouin Raoult, ECMWF, Reading, United kingdom; ECMWF, Reading, Berkshire, United kingdom; and M. Chantry, F. Pinault, J. Dramsch, and F. Pappenberger

5:30 PM
4A.4
Towards Comprehensive Evaluation of Data-Driven Numerical Weather Prediction Models
Jaideep Pathak, NVIDIA Corporation, Santa Clara, CA; Nvidia Corporation, Santa Clara, CA; and B. Bonev, T. Kurth, N. D. Brenowitz, Y. Cohen, K. Kashinath, J. Kossaifi, K. Azizzadenesheli, N. Kovachki, M. Baust, C. Hundt, A. Anandkumar, and M. Pritchard

5:45 PM
4A.5
Dynamical Tests of a Deep-Learning Weather Prediction Model
Gregory J. Hakim, University of Washington, Seattle, WA

- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner