89th American Meteorological Society Annual Meeting

Monday, 12 January 2009: 2:00 PM
Systematic Error Detection in Dynamical Models: A Framework
Room 126A (Phoenix Convention Center)
S. Lakshmivarahan, University of Oklahoma, Norman, OK; and J. Lewis
            It is the rule rather than the exception that operational weather prediction models exhibit systematic error. Over the Gulf of Mexico, the NCEP/EMC operational model has been plagued with systematic error in the marine-layer moisture for several decades — a negative bias from the late-1980s to the mid-1990s followed by a positive bias from the mid-1990s to the present. Despite a sustained effort by the research and operational community at NCEP/NHC, the biased forecast remains. The symptoms of the bias forecast can be ameliorated to a certain extent, i. e., from knowledge of the bias, the data assimilation scheme can empirically alter the analysis. Clearly, it is preferable to identify and remove the source of the error.            We have built a framework that searches for the source(s) of systematic error. We apply our methodology to the cool season weather regime that is linked to the systematic errors mentioned above — return flow over the Gulf where the consensus of opinion indicates that erroneous turbulent moisture/heat flux is the principal culprit. Rather than using the general operational model, a primitive equation model with a variety of parameterizations, we will use the mixed layer model that has been found to be appropriate for the generation of the convective boundary layer that forms during the outbound phase of return flow. The data we use were collected during the Gulf of Mexico Experiment (GUFMEX) in 1988. These data provide an exceptional depiction of the air mass transformation that took place as the air moved over the coastal waters toward the Loop Current in mid-Gulf.            Our framework follows:            Let  denote the state of a dynamical system at time . Let  and  denote a set of () parameters of which the first  components denote the initial conditions, i. e.,  for  and the last  components denote the physical parameters. Let  and                                                                                                                 (1)      

denote the model dynamics. To emphasize the dependence of the solution on the parameters , we explicitly denote the state system state as . Given the current operating point , the solution  denotes the model forecast.            Let  be a mapping such that                                                                                                                                                            (2)

denotes the model predicted observations.            It is tacitly assumed that the model  and the map  are exact, i. e., there are no errors in the model or in the relation that links observations to the model state.            Assume observations  are available at a specific time  where these observations are denoted by . The difference between observations and model counterparts to the observations is given by                                          .                            (3)

This constitutes a measure of the forecast error. Since  and  are assumed to be exact, a little reflection reveals that the forecast error is only due to errors or biases in the parameter vector . Given the forecast error in (3), our goal is to find a correction  to the parameter  such that  closely resembles.            We start by quantifying the first variation in  resulting from the perturbation of  by . To this end, first define the Jacobian, , of  with respect to , given by                                                         (4)

where . Then                                                                                          (5)

The  matrix  is also known as the first-order sensitivity matrix.            The variation  in  induces a variation  in  that is given by                                                                                       (6)

where the Jacobian  is an  matrix given by                                                                                   (7)

and . Combining (5) and (6), we obtain                                                      .                                       (8)

Given the operating point  and  in (3), we search for  such that                                                      .                                          (9)

Combining (8) and (9), given , we seek  such that                                                 .                                               (10)

Defining                                                   .                               (11)

Then (10) becomes                                                            .                                                        (12)            Since  in general, this equation is solved as a linear least squares problem. From Lewis et al. (2006), we get the solution as                                                        if                                (13)

or                                                        if .                              (14)

A number of observations are in order.

1. The matrix  defined in (11) is a product of two Jacobian matrices  and . Given , the matrix  can be explicitly computed. However, to compute the second factor , we need to know the dependence of  on . That is, we need to know the solution  of (1). Since (1) is in general non-linear, except for special cases, we cannot obtain the solution . Hence clever numerical schemes must be designed to compute the evolution of  with .

2. There is a vast literature dealing with the computation of first-order sensitivity matrices. The anthology edited by J. B. Cruz (1979) contains many of the classic results developed in the context of control theory. The review paper by Rabitz et al. (1983) contains a very readable account of the key results developed in the context of chemical kinetics. The recent monographs by Cacuci (2003) and Cacuci et al. (2005) contain an up to date coverage of topics related to sensitivity analysis.

3. Our goal is to demonstrate the utility of this framework. Accordingly, we have chosen the 3-variable nonlinear mixed layer model mentioned above. In a companion paper, we plan to illustrate the numerical process of computing  using well-known methods discussed in the literature.

4. Since we have the luxury of an exact solution in the mixed-layer model case, we also illustrate the computation of second-order sensitivity.

We have generated systematic error in the mixed layer model that mimics the results found in operational practice. The methodology outlined above has succeeded in identifying the sources of error in a variety of cases.

Supplementary URL: