Use of a nonlinear filter to improve the quality of Oklahoma Mesonet surface observations

The Oklahoma Mesonet uses a complex set of automated quality assurance tests to aid quality assurance meteorologists in detecting data problems in more than 800,000 observations that are collected daily by the Oklahoma Mesonet. The nonlinear filter was added to the suite of tests in 2009 to remove erroneous increases and decreases that were not being flagged by the spatial, like instrument or step tests. The filter was developed by Fred Brock (1986) and takes the median of the observation in question and the four observations temporally surrounding the observation in question. If the difference between the observation and the median is greater than a predetermined threshold, then the observation is removed from the Mesonet archive.

The filter helps remove noise that is often observed in these observations that tend to have relatively smooth time series (i.e., soil temperature, pressure, soil moisture). The test does not work well on time series that have a larger variance (i.e., wind speed).