An Audit of the National Center for Atmospheric Research (NCAR) Earth Observing Laboratory (EOL) Instrument Data Workflow from Ingest to Archival

Researchers often require specialized suites of instruments in order to carry out the scientific field work needed to understand the complex, interdependent geophysical processes that govern our planet. The National Science Foundation provides the National Center for Atmospheric Research (NCAR) Earth Observing Laboratory (EOL) funding for the acquisition, operation, maintenance and upgrade of these instruments to support observational meteorology across the globe. The data collected during these field campaigns are the legacy of this investment. Simple and timely community access to collected data and associated software tools, and preservation and quality control of the collected data for decades, fosters scientific research well past the end of the field phase of a project.

Support of EOL instrumentation is spread across three observational Facilities; the In-situ Sensing Facility (ISF), the Research Aviation Facility (RAF) and the Remote Sensing Facility (RSF). Data handling processes were developed independently in each of these Facilities, and are based on historical precedents, so data workflows vary widely. EOL's Computing, Data and Software (CDS) Facility has led the efforts to increase the coordination and stewardship of these data and processes across our lab. This data audit maps the data workflow of each instrument or other data source from acquisition to archive. Analysis of the information collected during the audit will provide confirmation that all data and associated software are safeguarded throughout the data handling process, and that community standards of practice for data stewardship and software version control are followed. This in turn assures that data and software are available for future use, and supports the reproducibility of published results. Additionally, this analysis will provide guidance for enhancements to preservation, access and use and facilitate the synchronization of data management efforts in each Facility, increasing efficiency, reducing redundancy, and preserving integrity, thus reducing costs.

The techniques used to perform this audit will be discussed, including types of information collected, analysis performed, and the types of hurdles that were encountered and how they were overcome.