The Arctic Observing Network and its Data Management Challenges: Three Years On

In 2007, the National Science Foundation funded the National Snow and Ice Data Center, the National Center for Atmospheric Research, and the University Corporation for Atmospheric Research to develop the Cooperative Arctic Data and Information Service (CADIS) for data from the nascent Arctic Observing Network (AON) of funded investigators. AON was envisioned as “…a system of atmospheric, land- and ocean-based environmental monitoring capabilities--from ocean buoys to satellites--that will significantly advance our observations of Arctic environmental conditions”. Its data service was envisioned as a portal through which scientists can archive their data and thereby meet NSF sponsor obligations; can find all data relevant to a location or process; can browse imagery and complete documentation; may plot online time series or fields; and can query multiple datasets and sources, which have all metadata stored in a relational database. We accepted the funding with eyes wide open, well aware of the insurmountable nature of some of the challenges involved but anxious to make progress. A talk on AON at the 2008 AMS IIPS session outlined the challenges and our anticipated approach. We faced multidisciplinary data in many formats, a diversity of archiving and access needs, and visionary investigator expectations. We would leverage other cyberinfrastructure and enterprise data management interoperability efforts.

Here we review what is working and what is not. From a technical standpoint, data set identifiers and ‘parent/child' relationships remain major hurdles in spite of the progress of the international community on this issue. Arriving at the specialized controlled vocabularies needed before semantic search can work remains out of reach, as does inventory level metadata or easy-to-use translators to get data into netCDF. Nonetheless, substantial progress has been made relative to our metadata challenges in CADIS. Our online metadata entry tool builds off others but is customized for AON and makes it easy for AON PIs to submit data and metadata. Standard metadata underlie advanced data search capabilities offered by our portal (aoncadis.ucar.edu), are compatible with GCMD, FGDC, ISO, IPY, and other international conventions, and have demonstrated interoperability for catalog sharing (via OAI-PMH). A survey of AON PIs confirmed that maintaining our responsive Help Desk will be key to user satisfaction as we continue CADIS development with a team of about 20 dedicated data managers and software engineers.