Using Machine Learning Techniques to Switch Between Distributions to Improve Data Assimilation

With the development of non-Gaussian based data assimilation in the variational formulation, and the understanding that the underlying distribution can change dynamically, we need techniques that allow us to “switch” between the distributions to ensure a consistent background error model. To address this question, we have used three different machine learning techniques with different criteria to determine if the behavior of the variable has switched between Gaussian, lognormal and reverse lognormal. In this presentation, we shall present results using machine learning algorithms to determine when the z component of the Lorenz 1963 model switches between Gaussian, lognormal, and reverse lognormal, through using skewness changing from 0 to determine this change and show that by switching between the distribution, the analysis error is improved compared to just assuming a Gaussian all the time. We also compare the efficiency of these methods and determine the size of the training data requiring for optimizing the accuracy of the machine learning techniques to detect changes in the underlying distribution. Using the optimal size of the training data, we implement both 3D and 4D variational methods using the cost functions defined by Gaussian, lognormal and reverse lognormal distribution.