84th AMS Annual Meeting

Monday, 12 January 2004
Merging NetCDF and HDF5
Hall 4AB
Russell K. Rew, UCAR, Boulder, CO; and E. J. Hartnett
Poster PDF (182.4 kB)
Unidata's netCDF data model, data access libraries, and machine independent format are widely used in the creation, access, and sharing of geoscience data. More recently, NCSA's HDF5 data model, libraries, and format have been used in NASA Earth Science programs and in large computational models that require parallel I/O. Unidata and NCSA are collaborating to merge netCDF and HDF5 to achieve gains in performance and interoperability.

If the merger is successful, users of netCDF will benefit from support for packed data, large datasets, and parallel I/O, and users of HDF5 will benefit from the availability of a simpler high-level interface suitable for array-oriented scientific data, wider use of the HDF5 data format, and more software for data management, analysis and visualization that has evolved among the large netCDF user community. The overall goal of this collaborative development project is to create and deploy software that will preserve the desirable common characteristics of netCDF and HDF5 while taking advantage of their separate strengths: the widespread use and simplicity of netCDF and the generality and performance of HDF5.

We summarize the current status of the work, including new software that uses the current netCDF interface with the HDF5 format to provide backward compatibility with current data and application software. We also describe enhanced netCDF-4 software under development and how it makes use of advanced features of HDF5.

Supplementary URL: http://www.unidata.ucar.edu/packages/netcdf/index.html