Overview of an HDF-EOS-based parallel data-computing environment for multisensor satellite data merger and scientific analysis

The Data Processing Error Analysis System (DPEAS) is a parallel data processing system for multisensor satellite data analysis. Current applications include near real-time transfer of AMSU data sets to the National Weather Service and the Satellite Analysis Branch. Typical data rates exceed 6 TB/yr using a 4 node (8 processor) PC cluster serving 5 simultaneous near-real-time satellite sensor feeds, including GOES-08, GOES-10, NOAA-15 AVHRR, NOAA-15 AMSU-A, and AMSU-B. The system’s internal memory structures mimic the HDF-EOS data format data structures, which facilitates data input and output tasks. The parallel processing environment utilitizes an interpreted fortran 95 script language requiring minimal user training. In many aspects, the inputs are like a fortran-based version of Java with parallel programming heuristics incorporated into the architecture. A web-based interface is used for remote access to the system including the data product files. The system is currently capable of merging 13 satellite platforms with 9 different sensors. Selected examples of the processing capabilities and design features will be shown.