Thursday, 17 January 2002: 11:44 AM
Updating and enhancing the quality control system for solar radiation data processed by the MSC
The Canadian solar radiation network presently consists of 43 stations distributed throughout the country. The radiation fields commonly measured at most stations include RF1 (global irradiance), RF2 (diffuse irradiance), RF3 (reflected irradiance), RF4 (net irradiance), and in some cases RF9 (incoming infrared irradiance). The network was established in the mid-1960's. From 1980 onwards, the size of the network has been shrinking and resources for inspection, maintenance and quality control have been cut back. Until the mid-1990's, Canadian solar radiation data was subjected to vigorous human interactive quality control procedures. The interactive quality control program used for this operation has been identified as a critical component in the processing of solar radiation data, but requires that all data be visually inspected and that the operator be a highly trained technician. At present, the quality control of solar radiation data is inadequately resourced and a backlog exists. To increase the efficiency in which solar radiation data are processed, there must be a coordinated effort to i) modernize data acquisition and transmission throughout the network, and ii) develop and enhance automated procedures to reduce the reliance on human data handling and interactive quality control. Network modernization has already begun and it is expected that data will be transmitted automatically over the wide area network in near real-time mode. In anticipation of the receipt of digital data in near real-time, the MSC is developing automated quality control procedures to facilitate the archiving of solar radiation data in a more timely fashion. Algorithms under development will assist in identifying potentially erroneous data and direct a quality control technician to the suspect period for visual inspection and detailed analysis. This will greatly accelerate the processing and quality control of backlogged data. Planned upgrades include the translation of the current interactive QC program into a modern graphical computer language, the addition of more QA/QC filters and algorithms (e.g. expanded range checks, step tests, persistence tests, like-instrument checks and spatial checks used in other networks like the Oklahoma Mesonetwork), generation of a "daily errors" report, automatic bridging of small gaps between data transmission periods, and the creation of a "common errors" database which will be populated via the interactive QC process. For stations that are transmitting data in near real-time (e.g. once a day), daily thumbnail graphs will be generated for quick inspection by a trained technician. From the visual analysis and the daily errors report, subsequent interactive quality control will be efficiently targeted. The daily graphs will also be posted on the MSC Intranet and made available to regional offices so they can monitor the performance of their stations and instruments. The development of the data processing and quality control systems proposed will make solar radiation data more readily accessible to clients and ultimately improve the quality of the measurements since errors due to maintenance issues will be detected and corrected on the order of days, rather than weeks or months.