Netflix to NOAA: Commercial best practices for Highly Available Architectures for mission critical weather systems

The general public relies on timely life-safety information from several federal agencies operating mission critical systems that gather information and get the word out ahead of hazardous weather. To achieve the high resiliency and availability required of these systems, agencies must often overcome technological hurdles using innovative solutions incubated across the public, academic, and private sectors. As a system integrator, Booz Allen has been exploring the use of highly resilient commercial design patterns and architectures to further help federal agencies adopt these innovations across the weather enterprise. When it comes to data processing and system uptime innovations for example, we looked at frameworks used by streaming movie companies, like Netflix, who process 1.3 petabytes of data across 500 billion data events every day. The viability of the streaming media business model is predicated on a 99.999% uptime availability of movies to millions of viewers any time of day. Resiliency, flexibility, and redundancy must be baked into these systems and architectures to account for the real-world failures that will inevitably happen. By combining an understanding of high availability and throughput designs with human-centered design (HCD), commercial companies employ systems architectural innovations that allow for the most critical services to remain operational while other non-essential services are quickly restored.

So, what does this have to do with the weather enterprise? NOAA processes high-volume data from and across LEO and GEO satellite systems, NEXRAD radar data, the Advanced Weather Interactive Processing System (AWIPS), and more. These mission critical systems across the weather enterprise can adopt these innovations to ensure its weather offices, emergency managers, and industry partners can still receive life-saving forecasts, data products, and alerts even when some systems fail. Using innovations in HCD thinking and highly available architectures (HAA) can maximize the resiliency of the most critical services of a system to meet the realities of real-world failures.

In this presentation, we will explore and demonstrate the use of open cloud-native platforms using HAA similar to what’s used by Netflix and other commercial companies for mission critical systems operating in the cloud and how this relates to the resilience of NOAA’s mission critical weather systems. We will demonstrate how designing innovative HAA infrastructure with the user in mind can achieve 99.999% reliability for mission critical systems to keep the public safe ahead of hazardous weather.