Implementing an enhanced and integrated quality assurance and quality control system within the MSC's new Data Management Framework

The Meteorological Service of Canada (MSC) is in the process of improving its electronic data management practices for weather, climate, hydrometric, air quality, and image data. A project is currently underway to implement a new Data Management Framework (DMF) which will provide the infrastructure and facility to collect, decode, transform, conduct quality assurance (QA) and quality control (QC), archive, generate/distribute data products, and provide a single point of entry for data access (i.e. from one logical system and integrated database) for a wide variety of users such as forecaster workstations, operational data assimilation systems, etc.

The ability of the MSC to perform modern and comprehensive data management and quality assurance on its data has been constrained by having multiple databases, formats, QA systems and standards, algorithms and flagging schemes across Canada. One of the main advantages of the proposed framework is the opportunity to harmonize many of the QA/QC systems and procedures currently practiced regionally and/or on a network-by-network basis. The DMF will streamline and simplify the incorporation of data from many different sources, types, networks, platforms, etc., to enable the automatic QA/QC system to better perform intervariable, temporal and spatial consistency tests, which were previously not possible.

A significant amount of data collected from Canada's surface weather and climate stations, including those in the GCOS network, are transmitted over the Global Telecommunications System (GTS) to the world in real-time. In many cases these products (e.g. SYNOP bulletins) have undergone little or no QC. Under the DMF, raw data collected will go through real-time automatic QA/QC before distribution or product generation. The ability of the DMF to have access to data from most of Canada's weather and climate networks in real-time and in one logical location allows for more sophisticated QA/QC as well as more efficient dataflow to numerical weather prediction models and other applications or products which rely on data from multiple sources.