Cloud-Based Machine Learning Capabilities to Improve Weather Event Predictions

Per Department of Commerce (DOC) and NOAA Office of the Chief Information Officer (OCIO) direction, NOAA has began initiating a heavy focus on transitioning applications and services into the cloud. With this comes the availability of cloud-hosted Machine Learning and Artificial Intelligence capabilities which can be leveraged to advance the exploitation of NOAA’s environmental science databases toward improving environmental and earth science predictions. The National Weather Service (NWS) weather community presents a huge potential target of opportunity for weather prediction-focused applications in the cloud. Other weather community partners such as NASA and even commercial companies could also benefit from such weather prediction capabilities in the cloud.

One such example, which Solers has formed an Internal Research & Development (IR&D) project around, is improving weather event predictions. Solers’ Weather Event Prediction (WEP) IR&D capability will provide a means of data mining live data feeds for weather events, specific to a NWS Weather Forecast Office (WFO)’s need, and using it to provide weather event prediction via machine learning with AWS cloud services. The initial target use case for the WEP IR&D project is improving flash flood predictions in areas that are historically prone to such events. Data mining and Machine Learning algorithms can be tailored for WFO-specific applications (i.e., the specific to the terrain of a WFO’s area). The WEP IR&D project uses NCEI’s Storm Events Database to pick a target NWS WFO (regional area) where significant flash flood related damage has occurred for the IR&D project scope. We then create a flash flood prediction capability in the Amazon Web Services (AWS) cloud that uses real-time and historical mined data feeds from external data sources, combined with machine learning, to predict the likelihood and damage severity/impact of flash flood events for the target WFO’s regional area. The mined data metadata/statistics and flash flood prediction results can be visualized via a web browser using graphical charts/graphs leveraging AWS cloud services.

For the WEP IR&D project implementation, Solers is mining meteorological observational data from the NWS NCEP MADIS database and placing it into Amazon Redshift for analytics queries during Machine Learning. We then mine storm reports data from the NWS Storm Predictions Center database and place into Amazon Redshift for analytics queries during Machine Learning. Next, we develop and train a Machine Learning algorithm leveraging mined data feeds from Amazon Redshift, using Amazon SageMaker and Amazon S3 for the Machine Learning algorithm development and training. We then execute the Machine Learning algorithm in Amazon Machine Learning, storing prediction results in Amazon Redshift. The data mining metadata/metrics and machine learning prediction results can then be visualized in Amazon QuickSight.