Migration to Cloud and Path to Modernization for the Joint Polar Satellite System Data Production System

The Joint Polar Satellite System (JPSS) Common Ground System (CGS) Interface Data Processing Segment (IDPS) is the key component in the National Oceanic and Atmospheric Administration (NOAA) Polar Satellite data processing for the National Weather Service. IDPS’s mission is to create extremely high-quality weather products with very low latency and very high reliability. IDPS was designed in the early 2000’s and the system design has matured over the past 15 years, but the initial concepts of modularity and flexibility have proven to be a nearly perfect fit for operation in cloud environments. In 2018 the JPSS Ground Project and Raytheon IDPS team performed successful proof-of-concept forklift migrations to both AWS and Azure. During these preliminary migration activities, a high level of capability was demonstrated with quick turnaround proof of concept studies: rapid deployment of infrastructure, live satellite data feed, multi-mission data production, data delivery to mission partners, maintenance release update and string transition. With the success of these migrations, NOAA has directed the JPSS CGS program to transition IDPS operations to AWS no later than EOY 2020.

IDPS migration to the cloud is planned in 3 stages: Initial Implementation, Optimization and Modernization. In the Initial Implementation phase, an accelerated path is taken to transition-to-ops with minimal baseline changes to meet deadline for decommissioning legacy hardware before EOY 2020. The Optimization Phase will follow with increased utilization of cloud-native services both to save costs and to simplify system architecture for securability and maintainability. This phase will take advantage of processing elasticity to obviate the need for “hot” backup processing systems while maintaining the high reliability and availability of data products. In the Modernization Phase the IDPS will take full advantage of cloud services and integrate new and modern services with the broader NOAA enterprise cloud initiative. In this final phase, re-architecting production and delivery subsystems will significantly decrease the research-to-ops (R2O) timeframe for algorithm changes which are critical to the forecasting and science communities. With these modifications we will also decrease the delivery time for ops data to the research community. This paper describes a phased approach to modernization and how the implementation of each phase will impact the NOAA science and weather forecasting community.