Physics-Informed Machine Learning Methods for Post-Processing Weather Elements

Physics-informed machine learning (PIML) has emerged as a rapidly developing research discipline that seeks to blend traditional scientific mechanistic modeling (differential equations) with machine learning (ML) techniques such as deep learning (DL). PIML achieves this integration through innovative methodological solutions to address domain-specific challenges and extract insights from scientific datasets. It aims to advance scientific understanding by producing generalizable, scientifically consistent and interpretable ML models, that are domain-aware, scalable, and robust. PIML has already been demonstrated to reduce the generalization error and prediction uncertainty for modeling complex dynamical systems [1].

Within the domains involving complex engineering and environmental systems, a growing interest of using PIML is evident from several workshops [2-8] and survey studies [9-15] physics-informed machine learning” on Web of Science shows an increase in the number of publications from 65 in 2020 to 301 in 2022, with 185 already published as of August 2023.

Physics-informed neural networks (PINNs) were introduced in [16] and demonstrated effectiveness in inferring data-driven solutions to general nonlinear PDEs and data-driven discovery of PDEs. Attributed to the modular and flexible nature of NNs, PINNs provide ample opportunities to add domain specific knowledge. [17] summarizes different methods and applications of PINNs across disciplines. Figure 1 shows architecture of a PINN.

Our study is a part of an ongoing work at Embry-Riddle Aeronautical University (ERAU), Daytona Beach campus, where we are developing a PIML model to forecast weather conditions that would potentially disrupt airport and airspace operations. We are developing a PIML model to post-process the forecasts made by the Global Forecast System (GFS) model, using weather observations at the airports from METAR data as ground truth. We intend to determine if GFS’s forecast skill can be improved through PIML, and then assess how well it can be. We also aim to compare the performance of state-of-the-art DL models like LSTM, ConvLSTM, and the more recently introduced Fourier Neural Operators (FNOs). Since the nature of input data helps determine the ML model to be used, it would be interesting to know how the nature of the input data impacts the performance of different models. Different For example, a long short-term model (LSTM) could be used if the weather data is only considered as a time series i.e., considering weather at 39 airports discretely, whereas a ConvLSTM could be used if weather data is considered as spatio-temporal i.e., each of the 39 airports are different points/places in the space. In our presentation at the conference, we intend to share the results and lessons learned. A brief introduction to the data processing, followed by the architecture of PIML models trained and tested to post-process the GFS predictions, and finally a performance comparison of these models shall be presented at the conference. Additionally, to add model explainability and interpretability of the results, we intend to explore and implement different methods prevalent in the ML community. A brief discussion on how such methods were employed in this study shall also be presented.

References

[1] Erichson, N. & Muehlebach, Michael & Mahoney, Michael. (2019). Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction.

[2] 2023. AI4Science Workshop: AI for Accelerating and Enhancing Scientific Simulations. https://www.ai4science.caltech.edu/events.html

[3] AI for Science. https://ai4sciencecommunity.github.io/

[4] 2023. Workshop on Scientific Machine Learning (SCIML). https://sites.utexas.edu/scimlworkshop/

[5] 2022. TrAC Workshop on Scientific Machine Learning: Foundations and Applications. https://trac-ai.iastate.edu/Activities/workshops/SciML2022/

[6] 2022. Scientific Machine Learning for Complex Systems: Beyond Forward Simulation to Inference and Optimization. https://www.santafe.edu/events/scientific-machine-learning-complex-systems-beyond-forward-simulation-inference-and-optimization

[7] 2022. CNLS Annual Conference 2022 - Physics Informed Machine Learning. https://web.cvent.com/event/ead9c1d8-c632-4896-ba9e-d5f3fec81e86/summary

[8] 2023. Machine Learning for Science & Engineering. https://dynamicsai.org/seminars/MLWorkshop/

[9] Kashinath, K., Mustafa, M., Albert, A., Wu, J. L., Jiang, C., Esmaeilzadeh, S., Azizzadenesheli, K., Wang, R., Chattopadhyay, A., Singh, A., Manepalli, A., Chirila, D., Yu, R., Walters, R., White, B., Xiao, H., Tchelepi, H. A., Marcus, P., Anandkumar, A., Hassanzadeh, P., ... Prabhat (2021). Physics-informed machine learning: case studies for weather and climate modelling. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 379(2194), 20200093. https://doi.org/10.1098/rsta.2020.0093

[10] Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. 2022. Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems. ACM Comput. Surv. 55, 4, Article 66 (April 2023), 37 pages. https://doi.org/10.1145/3514228

[11] Karniadakis, George & Kevrekidis, Yannis & Lu, Lu & Perdikaris, Paris & Wang, Sifan & Yang, Liu. (2021). Physics-informed machine learning. 1-19. 10.1038/s42254-021-00314-5.

[12] Meng, Chuizheng & Seo, Sungyong & Cao, Defu & Griesemer, Sam & Liu, Yan. (2022). When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning.

[13] Muther, Temoor & Dahaghi, Amirmasoud & Syed, Fahad Iqbal & Pham, Vuong. (2022). Physical laws meet machine intelligence: current developments and future directions. Artificial Intelligence Review. 56. 1-67. 10.1007/s10462-022-10329-8.

[14] Hao, Zhongkai & Liu, Songming & Zhang, Yichi & Ying, Chengyang & Feng, Yao & Su, Hang & Zhu, Jun. (2022). Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications. 10.48550/arXiv.2211.08064.

[15] Pateras, J.; Rana, P.; Ghosh, P. A Taxonomic Survey of Physics-Informed Machine Learning. Appl. Sci. 2023, 13, 6892. https://doi.org/10.3390/app13126892

[16] Raissi, Maziar & Perdikaris, Paris & Karniadakis, George. (2018). Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations. Journal of Computational Physics. 378. 10.1016/j.jcp.2018.10.045.

[17] Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. 2022. Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next. J. Sci. Comput. 92, 3 (Sep 2022). https://doi.org/10.1007/s10915-022-01939-z