This level of integration requires that data be in a limited number of formats or in self-describing formats. To be able to find the data, and use it in this way, data must have both high level (collection level) and use (inventory level) metadata. But AON data are quite heterogeneous: most projects are acquiring data that are in ASCII files of various configurations and few have complete metadata. Complicating the picture further, investigators want to find and use not only AON data but data from other sources as well without having to deal with a multitude of formats.
The Cooperative Arctic Data and Information Service (CADIS) will provide AON with near-real-time data delivery, a repository for data storage, a portal for data discovery, and tools to manipulate data by building on existing tools like the Unidata Integrated Data Viewer. Our approach to the data integration challenge starts by asking investigators to provide data in netCDF format that is compliant with the Climate and Forecast (CF) conventions. We also developed a "CADIS metadata profile" to guide the metadata structure of data contributed to the CADIS project. Structuring metadata according to this profile, and providing data in a limited set of standard formats, ensures that the metadata and data are immediately compatible with CADIS tools and can be visualized and manipulated online in various ways. Selected gridded or in situ data sets that are not in a standard format but are of high value to the AON community are converted to netCDF with special purpose software. A staffed Help Desk directs other users toward data conversion tools to encourage use of standard formats. An entry tool assists PIs in writing metadata and submitting data.
CADIS is a joint effort of the University Corporation for Atmospheric Research (UCAR), the National Snow and Ice Data Center (NSIDC), and the National Center for Atmospheric Research (NCAR). In the first year, we are concentrating on establishing metadata protocols that are compatible with international standards, and on demonstrating data submission, search and visualization tools with a subset of AON data. These capabilities will be expanded in years 2 and 3. By working with AON investigators and by using evolving conventions for in situ data formats as they mature, we hope to bring CADIS to the full level of data integration imagined by AON planners.
Supplementary URL: http://www.eol.ucar.edu/projects/aon-cadis/