A portion of the quality control process is done at the field sites. Preventative maintenance is done every 3 months at each field site. During these field visits, technicians will do vegetation maintenance, replacement/cleaning of sensors, as well as update our station metadata by collecting photographs of the area. Sensors that fail between preventative maintenance trips will be dealt with on a first come, first serve basis with strict documentation taken on to what was done and the nature of the sensor failure.
A majority of the quality control process occurs once the data reaches our database in Raleigh. Once inserted into the database, an automated series of QC algorithms are run to ensure data is accurate in the temporal and spatial dimensions. Temporal, or range, checks are parameter specific that compare the observation to a set of climatological values. Spatial, or buddy, checks are done after the range checks and check to see whether the valid range check values are comparable to values at nearby stations. Both range and buddy checks are then flagged from a set of values from pristine (R0,B0) to questionable (R2,B2) to invalid data (R4,B4). Data is given a score based on both the range check and buddy check to determine whether the data passes or fails the quality control check. Once data has been flagged, a visualization tool is used to edit any flags if a human believes the data is a valid value.
While the current QC routines do an adequate job capturing incorrect or failed data in our network, a few challenges still remain. The largest challenges involve the radiation sensors, precipitation events, and relative humidity. Future work involves new types of checks such as an intersensor check for parameters, a ratio check for solar radiation and photosynthetic active radiation, trend checks to catch sensor drift, radar based precipitation QC checks, and a new buddy check that uses spatial regression to weight neighboring stations in lieu of distance between stations.
Supplementary URL: