Using Forecast and Analysis Temporal Variability to Diagnose Model Performance and Predictability

Attributing forecast errors to initial condition errors or model errors is a difficult but worthy goal as it may guide forecast system development. Lorenz (1982) noted that the differences between forecast error growth rates and forecast difference growth rates are related to model deficiencies, and that as forecast models improve, these two growth rates should converge. Attempts to separate causes of the non-systematic component of forecast error include that of Dalcher and Kalnay (1987), building on the work of Leith (1978), in which it is assumed that the exponential part of the error growth is due to the self-growth in of initial condition errors, while the linear portion is due to model deficiencies.

Following these lines of investigation into model deficiencies, we use simple diagnostics to compare the temporal variability of forecasts as compared to that of analyses. We calculate the absolute differences for analyses that are i days apart (ai). We compare this to absolute differences between a forecast of length i days, and the analysis from which the forecast started (fi). In addition, we also calculate absolute differences between forecast of length i days, and the forecast of length i-1 days (di). These diagnostics are calculated for wind, geopotential height, temperature and moisture at every grid-point for 2 months of forecasts (January-February 2013), and averaged in time and space at various altitudes. We have first applied this diagnostic technique to the NCEP global ensemble forecasts (evolved from the pioneering work of Toth and Kalnay, 1993) in the TIGGE archive. We consider both the control (unperturbed) and perturbed members of the GFS ensemble.

In terms of geopotential height, the temporal variability for the forecasts and for the analyses are quite similar (that is, fi is close to ai for i=1,10). The 1-day temporal variability of the forecasts (di) usually decrease somewhat as i increases, but the changes for geopotential height are small, less than 10%. This indicates that, in terms of geopotential height, the model is behaving similarly to the atmosphere (as represented by analyses), and that the change in model forecast behavior with increasing forecast time is small. For other variables, the differences between fi and ai, and the changes in di with increasing i, are larger, indicating larger discrepancies between the forecast model behavior and the analyses, and larger changes in forecast model variability as forecast time increases. For example, for mid-latitude upper tropospheric temperatures, fi is about 15% smaller than ai for i=10 days, and di decreases by 30% between i=1 and i=10 days. In general, differences between the fi and ai tend to be larger for the perturbed ensemble member than for the control ensemble member. Spectral analysis of these differences indicates that most of the changes occur at synoptic and sub-synoptic scales. Ensembles forecasts from other operational centers are currently being examined and results will be interpreted in light of the different data assimilation and ensemble generation methodologies at the different centers.