Spatial Approaches to Evaluation of Climate Variability Projections and Predictions

Evaluations of the performance of earth system model predictions and projections on time scales ranging from subseasonal to seasonal (S2S) up to decadal and long-term climate are of critical importance to enhance usefulness of these products. Such evaluations need to address specific concerns depending on the time scales, system, and decisions of interest; hence, evaluation tools must be tailored to inform about specific characteristics of the forecasts. Traditional approaches – such as correlations, Brier scores and visual comparisons – which summarize comparisons of analysis and model grids, or between current and future climate, often do not reveal important information about the models’ performance (e.g., spatial or temporal displacements; the reason behind a poor score) and are unable to accommodate these specific information needs. For example, summary statistics such as the correlation coefficient or the mean-squared error provide minimal information to developers, users, and decision makers regarding what is “right” and “wrong” with a model.

New spatial and temporal-spatial object-based tools from the field of weather forecast verification (where comparisons typically focus on much finer temporal and spatial scales) have been adapted to more completely answer some of the important earth system model evaluation questions. In particular, the Method for Object-based Diagnostic Evaluation (MODE) tool and its temporal (three-dimensional) extension have been adapted for these evaluations. More specifically, these tools can be used to address spatial and temporal displacements in projections of El Nino-related precipitation and/or temperature anomalies, ITCZ-associated precipitation areas, atmospheric rivers, seasonal sea-ice extent, and other features of interest. Examples of several applications of these tools in a S2S or climate context (e.g., using the CESM large ensemble) will be presented. In general, these tools provide diagnostic information about model performance – accounting for spatial, temporal, and intensity differences – that cannot be achieved using traditional (scalar) model comparison approaches. Thus, they can provide more information that can be used in decision-making and planning. Future extensions and applications of these tools in a climate context will be described.