GOES-R Cloud-Hosted Products & Algorithm Verification Exercise

The GOES-R Product Readiness and Operations (PRO) team is supporting the task to move the ABI Level 2+ product generation (with cloud-native distribution capability folded in) to a cloud environment hosted in the NOAA/NESDIS Common Cloud Framework (NCCF). PRO has developed the Cloud-hosted Products & Algorithm Verification Exercise (Cloud-PAVE), which provides feedback on product format, structure, dimensions, metadata, correlation, and more to ensure an agile approach to handling issues in the data products produced by these cloud-native algorithm implementations.

Hypothetically, products produced in the cloud environment and the on-premises Ground System for the same time period should match exactly, given the input data to the algorithms should be the same. PRO uses the “Glance” tool from the University of Wisconsin and a custom metadata comparison script to compare these cloud generated files to their equivalents generated from the on-premises Ground System to ensure no harm is done to the product, or any discrepancies are understood. Differences are investigated thoroughly and either accepted because the reason is known, or sent back for algorithm reanalysis and reimplementation in the cloud environment.

The first stage of Cloud-PAVE compared files on an on-demand basis as each algorithm implementation was completed in the cloud environment. This method was sufficient for comparing products which did not require latent data or a spin-up period. PRO was satisfied with the results of these comparisons, with 66 of 82 products passing inspection. Of the products that did not pass, most were due to a known algorithm version difference between the cloud and Ground System environments. Only two failures were due to implementation issues.

This presentation will lay out the timeline to implement the ABI L2+ products in the cloud, results of the Cloud-PAVE effort, and will discuss the validation of products during the shadow-operations period in FY25.