Artificial Intelligence (AI) Techniques to Enhance Satellite Data Use for Nowcasting and NWP/Data Assimilation

Artificial Intelligence (AI), machine/deep learning techniques have advanced considerably in recent years across a number of areas and applications: in medicine, self-driving cars, social media, the finance industry, etc. The astonishing increase in accuracy and applicability of AI has been significant in the private sector, driven by the ease, efficiency, cost-effectiveness, speed and auto-learning features of AI. Significant advances have also been made in application of AI in different areas of meteorology and oceanography. However, until recently, far fewer AI applications were developed in the area of data exploitation of environmental satellite data, high-level information extraction in the area of numerical weather prediction (NWP), data assimilation and forecasting, as well as for extreme weather prediction and nowcasting. Based on the outcomes of the 2019 NOAA Workshop on AI for Earth Observation and NWP, there have been encouraging signs that AI is increasingly considered for these applications, with promising results, including predictive skills, and this trend is expected to continue with the significant challenges of ever-increasing volume of satellite data and the increased societal reliance on improved forecasting accuracy and resolutions. The increase of data volume comes from higher resolution satellites and sensors, from a growing list of new sensors (traditional as well as smallsats/cubesats), and from an explosion of new virtual observing systems made possible by the internet of things (IoT). Exploiting all these data sources is expected to present major challenges, and AI has emerged as a potentially transformational and mitigating technology. Meta-transfer learning, the transfer of knowledge and expertise from fields in which AI has been firmly established, to NWP and related environmental sciences, is found to be a powerful way to rapidly make significant advances. Partnerships with the academic and private sectors are also seen as key to achieve progress.

We will show results obtained when using AI for several critical areas important for data assimilation: satellite data calibration, forward operator simulation through radiative transfer, geophysical inversion, data assimilation and fusion, as well as for post-forecast correction, including for extreme weather events. For example, we will present recent results from a pilot project (MIIDAPS-AI), funded by several NOAA Research & Technology (R&T) programs, that retreives geophysical parameters from sensor radiances. MIIDAPS-AI is now undergoing the R2O transition. We will also demonstrate that the potential for AI techniques to infuse significant efficiency and enhanced skills into data assimilation is significant and merits consideration in mid-to-long term planning. We will also highlight a few challenges that need to be addressed for this technique prior to gaining recognition. Finally, we invite inputs, recommendations, and collaborations to advance NOAA's AI strategy.