87th AMS Annual Meeting

Monday, 15 January 2007: 11:30 AM
Who needs Sudoku when you have a puzzle like Data Management?
217A (Henry B. Gonzalez Convention Center)
Dave Spiegl, MSC, Toronto, ON, Canada; and P. Wong, R. Campbell, O. Jacobsen, and B. Fehr
Poster PDF (26.3 kB)
The Meteorological Service of Canada (MSC) collects a vast array of irreplaceable data. Handling of data has always been a core strength of the MSC, but the evolution of technology and the ever-increasing availability and volume of new data warranted significant changes to the way MSC manages these data. With an emphasis on ensuring data quality, MSC is tackling the way it handles data and metadata with a multi-year, life-cycle managed data infrastructure initiative called the DMF or Data Management Framework. The DMF rationalizes the flow of monitoring data throughout MSC's infrastructure to ensure that internal and external clients as well as MSC partners have reliable and relevant access to meteorological, hydrometric and air quality data and metadata for their immediate (real-time) and future (archived) requirements.

Some key data characteristics driving the Data Management Framework:

  • Existence of an official value for all MSC data
  • Inclusion of data history
  • Consistent real-time QA/QC
  • Data reliability and availability
  • Long term archiving of data and metadata
  • Improved access to MSC data sets
  • No data discarded
  • Implementation of a common data access policy
  • Using an iterative development plan, the DMF is well on its way. By early 2006, the DMF had delivered its first operational system for the management of data from the Road Weather Information Network (RWIN). This system serves as the central repository for observational data collected by environmental sensors installed within and beside roads throughout the provinces and territories of Canada and ensures that the data and associated metadata are quality controlled and archived securely. The DMF delivers data in real time to provincial transportation agencies for use in forecasting pavement temperatures and conditions. The MSC has also been involved in discussions with its counterparts in the United States to ensure the standardization of road weather data, and the creation of a seamless North American system.

    The focus of the most recent release of the DMF has been adjusted to support the Forecaster (NinJo) and Hydrometric workstation projects as well as the data sources originally identified in the DMF roadmap (i.e. Surface Weather, Aviation weather and Upper Air). Testing of the system and the resulting feedback from both workstation projects will provide valuable lessons for the DMF team. Successes with this release will encourage and facilitate MSC application developers to start converting applications to use the DMF in parallel with existing MSC programs.

Supplementary URL: