Migration of GOES-R Level 2 Processing Stream to an Automated Multi-Nodal Cloud Based Environment for Regional Level 2+ Data Processing

In this work, we describe the migration of a GOES-R Level 2+ product processing stream to a scalable cloud platform. This system is targeted at evaluating product generation and distribution systems for regional products in both on-demand and continuous processing paradigms. This prototype system includes the complete GOES-R Level 2+ baseline, as well as several tailored products, and is designed to be extensible to allow user-specific algorithms to be processed in the same environment.

The Geostationary Operational Environmental Satellite (GOES) series R (GOES-R), launched in late 2016 as GOES-16 and place into operations as GOES-East in late 2017, provides remote sensing data that represents significant advances over previous GOES missions. GOES-16 provides advanced Level 2 products based on government-supplied algorithms that describe the state of the atmosphere, land, and oceans over the Western Hemisphere with three times the spectral bands, four times the spatial information, and five times the temporal sampling over the previous GOES series. The effective net increase in L1b data throughput is over 60x with a similar increase in derived L2+ products. This presents significant challenges for data ingest, product processing and data distribution. This work, leverages prior efforts of the GOES-R Ground System (GS) Team who developed and delivered a robust and flexible system for producing and distributing these data products, as well as a comprehensive set of algorithm test tools that provide a user-oriented framework for testing. This companion framework, the Algorithm WorkBench (AWB), has been to augmented to serve as a light-weight framework for local and/or remote real-time region production of GOES-R baseline products as well as user defined tailored regional products.

This presentation provides an overview of the AWB architecture and the modifications made to incorporate this framework into a multi-nodal cloud-based environment. In this work, we focus on the Level 2+ algorithms that use sensor data from the GOES-R Advanced Baseline Imager (ABI), and provide observed throughput statistics and comparisons to GOES-R production products given mesoscale, CONUS and full disk scenes. Finally, we examine the application of this product production chain to similar to data sets including those from Advanced Himawari Imager (AHI) and Meteosat Second Generation (MSG) and Meteosat Third Generation (MTG).