A major goal of ensemble forecasting is to provide probabilistic forecasts. If by design, each ensemble member represents an independent sample from the true forecast probability density function (PDF), verifying analysis data will fall equally into each of the bins pooled and sorted from these ensemble members, i.e. the rank histogram is uniform. Then it is straightforward to use the ensemble for reliable probabilistic forecasting.

However, by construction, optimal perturbation ensemble generation techniques produce perturbations that point in the directions of the leading eigenvectors of the forecast error covariance matrix and have magnitudes equal to the square roots of their corresponding eigenvalues. Such perturbations do not represent random samples from the PDF. They represent the variance of an infinite number of random samples in a few key directions. By hypothesizing the functional relationship between these variances and the PDF, one can still attempt to construct probabilistic forecasts from first principles. However, since deficiencies in our knowledge of analysis error covariance and model errors make it impossible for us to construct optimal perturbation ensembles that precisely represent the leading directions of the forecast error covariance matrix, it is inevitable that the probabilistic forecasts constructed from first principles will have to be statistically corrected with verification data. The purpose of this talk is two fold. First, to describe how probabilistic forecasts can be constructed from first principles and, second, to show how these probabilistic forecasts can be statistically corrected with verification data.

In our scheme, eigenvalues and eigenvectors are used to construct a first guess of the PDF for forecast errors. Equal probability bins are constructed from the first guess PDF and a check is made to see how often the projected verifying analysis falls into each of these bins. Experiments are performed for both local and global verification regions. Typically, it is found that the projected verifying analysis falls into bins corresponding to extreme values more frequently than the central bins; i.e. the rank histogram has a U shape. Furthermore, there is typically a bias present. The data obtained from these experiments are then used to remove the bias of the ensemble and also to identify the constant factor that the eigenvalues of the PDF must be increased in order to obtain a flat rank histogram.

In our experiment, our approximate optimal perturbations are obtained by recycling or breeding perturbations using the Ensemble Transform Kalman Filter(ET KF) with a crude estimate of an analysis error covariance matrix. While the technique allows for the possibility of non-normal PDFs, to simply illustrate the technique we assume that forecast errors are normally distributed in the directions of the ensemble based forecast error covariance matrix.

Based upon this scheme, real-time, reliable probabilistic forecast products can be produced for regions of interest.