Extended Abstract (124K)7C.1
Meteorological Assimilation Data Ingest System Transition Project Risk Reduction Activity
James A. McNitt, NOAA NWS, Silver Spring, MD; and J. Facundo and J. O'Sullivan
The Meteorological Assimilation Data Ingest System (MADIS) is a data assimilation and distribution system developed by the National Oceanic and Atmospheric (NOAA) Office of Atmospheric Research to perform the functions of data ingest, data integration, processing for quality control, observation network monitoring, and data distribution for NOAA and non-NOAA observations worldwide. MADIS processes many types of meteorological data: surface observations from mesonets; upper air soundings, including balloon-borne and aircraft; remotely sensed data from both ground-based and satellite systems; and aircraft data. MADIS has reached a point where it can now be transitioned to operations at the NOAA's National Weather Service (NWS) Telecommunications Operations Center (TOC) in Silver Spring, Maryland during fiscal year 2009. NOAA's NWS initiated its risk reduction activity during 2006. This paper describes the project risk reduction strategy and the results achieved during 2007.
NOAA's NWS Office of Operational Systems Field Systems Operations Center (FSOC) identified Blade cluster technology as a key component of MADIS Transition architecture and conducted market research in 2005 and 2006 that confirmed the utility of this technology for use in the MADIS Transition. One benefit of Blade cluster technology is that it provides an affordable means for scaling and expanding the system to meet future requirements. Risk reduction activity conducted in 2007 showed the benefits to NOAA of using this technology to satisfy NOAA's real-time observational data processing requirements. The following performance areas were demonstrated using a structure testing approach: application failover, storage failover and database failover for system reliability, authentication and authorization, storage, system monitoring, system image provisioning, database monitoring and support, and virtualization.
Within the MADIS Transition conceptual architecture, software on the Blades will process multiple “data processing threads.” A thread represents the end-to-end processing of a specific dataset. The following functions are required to process a thread: data ingest, data decoding, data processing, quality control, temporary data storage, data monitoring, and data delivery. The FSOC's technical support team started testing a data processing thread in a HP cluster environment using virtualization software in mid-2007. Where the initial risk reduction activity focused on the hardware solution the second risk reduction activity is designed to show that the MADIS software developed by NOAA's Earth System Research Laboratory/Global Systems Division could be hosted within the MADIS Transition architecture as a data processing thread. Successful completion of the MADIS Transition Project risk reduction activity has provided an early demonstration of the benefits to NOAA and the Nation of transitioning the research capability developed by NOAA to architecture well suited for operations within NOAA's NWS Telecommunications Operations Center.
Session 7C, Challenges in Data Access, Distribution, and Use - Part II
Wednesday, 23 January 2008, 4:15 PM-5:30 PM, 207
Next paper
Recorded presentation