Recorded presentation4B.2
The Arctic Observing Network and its Data Management Challenges
Florence Fetterer, NSIDC/CIRES/Univ. of Colorado, Boulder, CO; and J. A. Moore
The Arctic Observing Network (AON) is a developing system of 34 land, atmosphere and ocean observation programs, some already operating and some newly funded by the National Science Foundation. This International Polar Year initiative is acquiring much of the data needed to drive the interagency Study of Environmental Arctic Change (SEARCH). AON will succeed in supporting the science envisioned by its planners only if it functions as a system and not as a collection of independent observation programs. Data management plays an essential role. AON planners envision an ideal data management system that includes a portal through which scientists can find all data relevant to a location or process; all data have browse imagery and complete documentation; time series or fields can be plotted online, and all metadata are in a relational database so that multiple data sets and sources can be queried.
This level of integration requires that data be in a limited number of formats or in self-describing formats. To be able to find the data, and use it in this way, data must have both high level (collection level) and use (inventory level) metadata. But AON data are quite heterogeneous: most projects are acquiring data that are in ASCII files of various configurations and few have complete metadata. Complicating the picture further, investigators want to find and use not only AON data but data from other sources as well without having to deal with a multitude of formats.
The Cooperative Arctic Data and Information Service (CADIS) will provide AON with near-real-time data delivery, a repository for data storage, a portal for data discovery, and tools to manipulate data by building on existing tools like the Unidata Integrated Data Viewer. Our approach to the data integration challenge starts by asking investigators to provide data in netCDF format that is compliant with the Climate and Forecast (CF) conventions. We also developed a "CADIS metadata profile" to guide the metadata structure of data contributed to the CADIS project. Structuring metadata according to this profile, and providing data in a limited set of standard formats, ensures that the metadata and data are immediately compatible with CADIS tools and can be visualized and manipulated online in various ways. Selected gridded or in situ data sets that are not in a standard format but are of high value to the AON community are converted to netCDF with special purpose software. A staffed Help Desk directs other users toward data conversion tools to encourage use of standard formats. An entry tool assists PIs in writing metadata and submitting data.
CADIS is a joint effort of the University Corporation for Atmospheric Research (UCAR), the National Snow and Ice Data Center (NSIDC), and the National Center for Atmospheric Research (NCAR). In the first year, we are concentrating on establishing metadata protocols that are compatible with international standards, and on demonstrating data submission, search and visualization tools with a subset of AON data. These capabilities will be expanded in years 2 and 3. By working with AON investigators and by using evolving conventions for in situ data formats as they mature, we hope to bring CADIS to the full level of data integration imagined by AON planners.
Supplementary URL: http://www.eol.ucar.edu/projects/aon-cadis/
Session 4B, Challenges in Data Access, Distribution, and Use - Part I
Tuesday, 22 January 2008, 11:00 AM-12:00 PM, 207
Previous paper Next paper