GOES-R Algorithm Library Services, an extensible framework for executing next generation environmental data products

GOES-R, the next generation of the National Oceanic and Atmospheric Administration's (NOAA) Geostationary Operational Environmental Satellite (GOES) System, represents a new technological era in operational geostationary environmental satellite systems. Along with the vastly improved scientific capabilities of the satellite itself, the GOES-R ground processing system is designed to provide a unified environment where products are generated in a data-driven fashion. Before scientific algorithms are integrated into this high-performance operational environment, they are developed in a pre-operational environment, where they can be scientifically vetted without the intense demands of a real-time, 24x7 system. In order to smoothly transition algorithms between these two environments, a framework must be developed to isolate algorithms from the specific constraints of each. The GOES-R Product Generation (PG) Library Services component was conceived to meet this need. The GOES-R Library Services provides a uniform framework in which algorithms can be executed, allowing the scientific aspects of product generation to be insulated from operational concerns such as data transmission, block processing size, and number of processing cores. Scientific algorithms wrapped in this framework can be instructed to run on a variety of block sizes, and separate blocks can be allocated to any number of cores. Each algorithm is provided with a generic Data Model Interface (DMI), which allows the scientific algorithms themselves to determine the data that they must read and write, while still providing operational flexibility for how these data are persisted and transmitted. This work presents a detailed design view of the Library Services component, and discusses how this framework provides a high-performance, flexible computing environment for algorithms and a seamless mechanism for transitioning GOES-R Level 1b and Level 2+ algorithms into operations.