Assured Operations of the Environmental Information Enterprise

U.S. agencies are accomplishing their missions to protect lives and property using complex systems comprised of hardware, software, networks, and human-machine interfaces. Automation of these systems is increasingly required to transfer information between enterprise system components at the speed of need. NOAA environmental systems generate data at rates of >20TB per day that are processed through the data chain and delivered to provide decision support to people serving in roles from air traffic controllers to emergency managers.

State-of-health and state-of-performance are currently conducted on select systems within the Environmental Enterprise Value Chain using a range of tools, techniques, processes, and people. There is value in a holistic, enterprise-wide approach to verify performance of systems along the value chain, as well as impacts of degraded performance at any point in the system of systems to the overall mission of protecting lives and property.

This paper will discuss prototyping efforts which intake large volume and velocity environmental data streams and observatory telemetry and perform descriptive and prescriptive analytics, while identifying and presenting anomalous behavior using a configurable dashboard. Having learned nominal data stream behaviors, the system detects and predicts future anomalies and downstream impacts. The system operates on three scales:

• Real-Time Information Assurance – Monitor spacecraft and ground system telemetry to perform real-time diagnostics of mission performance through the value chain. Uses data analytics to assess anomalies and trends for proactive enterprise system operations management. Perform corrective action to prevent disruptive degradation or outages impacting the operational mission.
• Operational Impact Assessment – Leverage forecast sensitivity to observation systems impacts (FSOI) using data analytics and machine learning to assess and monitor quality of products in response to detected anomalies.
• Long-Term Enterprise Performance – Improve NOAA observation systems resilience over time by assessing potential impacts on enterprise integrity to meet operational mission to provide warnings, watches and other environmental information.

Using a common system for three scales opens the possibility for exploring and applying mitigation techniques related to anomalies. This paper will discuss secondary data sources and algorithms that hold promise in providing gracefully degraded forecasts when primary data is missing or corrupted. Preliminary work on the application of data analytical techniques to tropical storm track and intensity prediction indicates that machine learning techniques can produce useful predictions of future storm directions based on comparisons with past tracks and environmental conditions, and use a smaller set of inputs with lower likelihood for data outages/corruption compared to typical physical models for storm track. The situation with respect to storm intensity is even more promising, as statistical techniques are competitive with physical models for accuracy; we will discuss some new potential data sources that have become practical with the advent of high-volume data processing.