Use of a Parallel Data Processing and Error Analysis System (DPEAS) for Transition of Multisatellite Hydrometeorological Products into Operations

The CSU/NOAA Data Processing and Error Analysis System (DPEAS) blends multiple satellite and model data sets within a single consistent software framework for research use and operational implementations. DPEAS capabilities include: (1) massive parallelization, (2) high level module/data fabric abstractions, (3) generalized data fusion and data assimilation algorithms, (4) automatic source code checks, and (5) real-time file replication for enhanced throughput. In the past, DPEAS has been ported from Windows to the AIX computer operating system for NESDIS operations. The Linux version of DPEAS 3.0 was created under a US Air Force-funded activity related to its use for cloud advection and analysis. The most recent DPEAS version 3.0 capabilities include the addition of native netCDF4 data handling and cloud advection capabilities, thus enabling tight links to the many NESDIS operational satellite data flows. The DPEAS Linux port greatly enhances the usability for NOAA and AFWA. The system is well suited to high computational throughput processing on the major satellite production systems, as well as interfacing to novel new private-sector cloud computing developments.

Recently, DPEAS has been interfaced to the aWhere, Inc. WeatherTerrainTM cloud-computing location intelligent data provisioning platform. This capability enables the near real-time distribution of high-resolution real-time multisatellite blended precipitation products to African Govt. ministries, Non-Governmental Organizations (NGOs) within the food security and public health domains, and other Bill and Melinda Gates Foundation (BMGF) collaborators to enable wide-spread use of the NOAA satellite data sets. To serve this need, real-time precipitation pilot studies using DPEAS for both Africa and Mexico/Central America regions are underway. This work also includes advanced dynamic intercalibration of the satellite products for real-time operational use in both the U.S. Govt. and in the public and private sectors. This leverages NOAA products and developments for wider Govt. use, including novel multi-disciplinary activities that require access to advanced multisensor capabilities.

In this work, we highlight (1) the new US Air Force-funded DPEAS-Linux capabilities, as well (2) new NOAA-funded Soumi-NPP and METOP-B satellite data algorithm product additions, as well as (3) examples of the DPEAS/aWhere interface enabling advanced cloud computing data provisioning services via the aWhere, Inc. platform for the BMGF-supported user community.

This research was supported by grants from the Air Force Weather Agency (AFWA) to the DoD Center for Geosciences / Atmospheric Research (CG/AR) at Colorado State University, the NOAA/NESDIS Product System Development and Implementation (PSDI) program, NASA Short-term Prediction Research and Transition Center (SPoRT), and the Bill and Melinda Gates Foundation (BMGF).