25th Conference on International Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology

11A.1

Efficient reduction and compression of weather radar data in universal format

W. David Pan, University of Alabama in Huntsville, Huntsville, AL; and P. R. Harasti, M. D. Frost, Q. Zhao, J. Cook, T. Maese, and L. Wagner

The Naval Research Laboratory (NRL) Marine Meteorology Division, Monterey, CA is developing a weather radar data assimilation system to enhance the safety of at-sea ship and aircraft operations, by assimilating radar observations of atmospheric environmental parameters into the Navy's Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS®) to improve the prediction of short term changes in in-situ hazardous weather, and to provide decision makers with products to help exploit or mitigate those changes.  The system will take advantage of Navy vessels having weather radar capability (e.g., SPS-48E/G: Hazardous Weather Detection and Display Capability; SPY-1 Tactical Environmental Processor).  The ships in the battle fleet having this capability will have the capability to each generate a full-resolution Universal Format (UF) radar data file approximately every 5 minutes. For the purposes of radar data assimilation, three files will be transmitted to the Fleet Numerical Oceanography and Meteorology Center (FNMOC), Monterey, CA in near-real-time per hour.  UF file sizes range from ~5 MB (SPS-48E) to ~13 MB (SPY-1).  To minimize the load on the operational bandwidth, the UF files have to be significantly compressed before transmission, requiring a high compression rate that even the state-of-the-art data compressors cannot provide.

To overcome this challenge, we analyzed archived SPS-48E UF data obtained from an at-sea experiment onboard the USS PELELIU (LHA5) in February 2006.  This data set contained a wide range of precipitation echoes spanning 22 hours of observations. We employed the divide-and-conquer approach on these UF files by separating the actual radar data from the headers, which account for about 30% of the overall UF file size. By efficiently exploiting the inherent redundancies existing in the headers, we are able to losslessly compress the headers to negligibly small sizes. A data thresholding operation was also adopted to achieve data reduction and thus larger compression on the radar data, while maintaining sufficient information content to impact analyses and forecasts at FNMOC with the COAMPS®.  As a final step, the open-source bzip2 algorithms were applied to the data. Depending on the amount of precipitation echo observed throughout SPS-48E radar area coverage, and on the amount of thresholding employed, we were able to compress the UF files down to between 88 KB and 500 KB.  The average compression ratio was approximately 20:1, which is the typical compression ratio expected for future SPS-48G and SPY-1 UF data as well.  The UF file compression software package will be tested for approval for ship installation by the Integrated Test Facility laboratory at the Space and Naval Warfare Systems Center (SPAWAR), San Diego, CA.

 

COAMPS® is a registered trademark of the Naval Research Laboratory.

extended abstract  Extended Abstract (760K)

Session 11A, Challenges in Data Access, Distribution, and Use - Part II
Wednesday, 14 January 2009, 4:00 PM-5:30 PM, Room 121BC

Next paper

Browse or search entire meeting

AMS Home Page