A Particle Flow Data Assimilation Method for High-Dimensional Systems

Many geophysical systems are highly nonlinear, asking for nonlinear data-assimilation methods.
Particle filters, after a rough start, are slowly becoming mainstream, evidenced by the recent
implementation by DWD of a localized particle filter for global numerical weather prediction.
However, as all data-assimilation methods at this scale of operation, localized particle filters need ad-hoc
tricks to avoid weight collapse, such as setting a minimal weight, or projecting observations onto the
ensemble space. Particle Flows are particle filters in which the samples from the prior are transformed via a
flow in state space to samples from the posterior, without the need for introducing weights.
Hence Particle Flows are equal-weight particle filters by construction. However, using them with
a small number of particles remains challenging.

By exploring ideas from machine learning, such as kernel embedding, kernel gradient estimation
and stochastic gradient descent we develop a new method that is extremely promising for
high-dimensional applications with a small number of particles. We will report on applying
this technique to a high-dimensional atmospheric example, discussing details of the
implementation, results, and remaining issues. These are exciting times for nonlinear data assimilation
and this contribution is meant to be a strong contribution to that effort.