Evaluation of an innovation variance methodology for real-time data reduction of satellite data streams

Despite the temporal gaps, polar orbiting satellites provide dense coverage over the otherwise data sparse ocean. At any given time, global data assimilation systems are faced with the overwhelming task of ingesting large volumes of disparate data. Driven, in part, by the necessity for efficiency under the constraints of real-time weather prediction, data reduction is common in the operational community. There are numerous approaches to the data thinning issue, which include quality control methods, observation subsampling, and the creation of superobservations. These methodologies do not necessarily guarantee that features—such as gradients—will be correctly resolved in the thinned data set. Here we attempt to quantitatively evaluate the impact of a systematic and efficient data removal methodology by testing three thinning algorithms: a subsampling method that takes every 7th observation, a box variance method that identifies redundant data by the way of a spatial estimate of the innovation variance (i.e., squared differences between the background field and observations), and an F-test variance method. The error characteristics of the thinned data are examined in the context of idealized (synthetic) analyses.