A Quantile Conserving Ensemble Filtering Framework for Tracer Concentration and Source Estimation and Parameters

Ensemble Kalman filter assimilation algorithms can be described as two steps: 1). Computing increments for an observed variable given an ensemble prior and an observation; 2). Computing increments for state variables given the observation increments. A novel efficient algorithm that allows the use of arbitrary continuous priors and likelihoods for the first step was presented at AMS 2022. The key innovation was to select posterior ensemble members with the same quantiles with respect to the continuous posterior distribution as the prior ensemble had with respect to the prior continuous distribution. At AMS 2023, a method for doing the second step via regression in a transformed space was presented in a poster. Observed and state variables were independently transformed with a probability integral transform followed by a probit transform, a specific form of anamorphosis. This method guarantees that the posterior ensembles for state variables have many of the advantages of the observation space quantile conserving posteriors. For example, if state variables are bounded then posterior ensembles will respect those bounds and eliminate most bias near the boundary.

Additional challenges arise when applying filters for quantities like rainfall or tracer concentration and source where the appropriate priors can be best represented by mixed PDFs, PDFs that are a sum of continuous and discrete functions. For instance, a source may be exactly zero with finite probability and it may be appropriate to have ensembles that have multiple members that are identical values. An extension of the quantile conserving filter framework that supports ensembles with duplicate values is presented. In addition to traditional verification metrics, an extension of the rank histogram for ensembles with duplicate values is presented. The new method produces significantly enhanced capabilities for tracer concentration and source estimation in an idealized tracer transport model and in atmospheric GCMs with chemical weather extensions. It is also appropriate for many parameter estimation applications.