Verification and Analysis of Aviation Jetstream Forecast from Artificial Intelligence Models

Abstract

The Hong Kong Observatory (HKO) has been implementing real time production of various Artificial Intelligence (AI) data-driven models for operational weather forecast applications. The AI models include (i) Pangu-Weather (Bi et al., 2023) from Huawei; (ii) Fuxi model (Chen L. et al., 2023) from AI Innovation and Incubation Institute, Fudan University Shanghai, and (iii) Fengwu model (Chen K. et al., 2023) from Shanghai Artificial Intelligence Laboratory. To evaluate the performance of these AI models on aviation applications, relevant real time aviation forecast products are developed. In this study, the aviation jetstream forecast from AI models, which is essential for flight planning and fuel optimization, is evaluated against ECMWF Reanalysis v5 (ERA5) and compared with ECMWF Integrated Forecasting System (IFS), a traditional physical Numerical Weather Prediction model. For fair comparison, the jetstream forecasts from various models have been remapped to the same resolution before performing the verification. Both grid-based verification and object-based verification are performed to evaluate not just the magnitude but also the shape and coverage of the jetstream forecasts. The study reveals that the AI models outperformed the traditional physical NWP model in most aspects, including grid-based error analysis and object-based verification metrics, with Fengwu being the most superior. However, the physical NWP model is able to capture finer details of the jetstream with higher magnitudes, hinting AI models may tend to produce overly smooth results. The study contributes to a better understanding of the usefulness of AI models as guidance in operational aviation weather forecasting.

Reference

Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. (2023). Accurate medium-range global weather forecasting with 3D neural networks. Nature, 619(7970), 533-538.

Chen, K., Han, T., Gong, J., Bai, L., Ling, F., Luo, J. J., ... & Ouyang, W. (2023). Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead. arXiv preprint arXiv:2304.02948.

Chen, L., Zhong, X., Zhang, F., Cheng, Y., Xu, Y., Qi, Y., & Li, H. (2023). FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6(1), 190.