Description:

Numerical weather prediction (NWP) models play a critical role in forecasting weather phenomena, guiding disaster preparedness, and informing decision-making processes. With the increasing adoption of artificial intelligence (AI) techniques in NWP, there is a growing need to develop rigorous testing, metrics, and validation frameworks to assess the performance and reliability of AI-driven models. This session aims to bring together researchers to discuss innovative approaches and best practices for evaluating AI in numerical weather prediction in a co-production environment. We invite contributions that address the following topics:

• Testing Frameworks: What methodologies and procedures can be employed to systematically evaluate the performance of AI algorithms in numerical weather prediction? How can we design comprehensive testing frameworks that account for diverse weather patterns, spatial-temporal scales, and model configurations?
• Metrics for Evaluation: What metrics and performance indicators are suitable for assessing the accuracy, reliability, and skill of AI-based NWP models? How can we develop standardized metrics that capture the strengths and limitations of AI techniques in capturing complex atmospheric dynamics and phenomena?
• Validation Techniques: How can we validate AI-driven NWP models against observational data, benchmark models, and ensemble forecasts? What strategies can be used to quantify uncertainties and assess the robustness of AI-based predictions under different environmental conditions and forecast lead times? How might established metrics for traditional physics based approaches be updated to incorporate AI?
• Case Studies and Applications: What insights can be gained from real-world case studies and applications of AI in numerical weather prediction? How have AI techniques enhanced forecasting accuracy, lead time, and spatial resolution in specific weather events or regions? What lessons learned and best practices can be derived from these experiences? What AI-ready data are needed to drive future data-centric AI development for NWP?