Challenges in Regional Verification of Climate Variability Projections and Predictions

Evaluations of the performance of earth system model predictions and projections are of critical importance to enhance usefulness of these products. Such evaluations need to address specific concerns depending on the system and decisions of interest; hence, evaluation tools must be tailored to inform about specific issues. Traditional approaches that summarize grid-based comparisons of analyses and models, or between current and future climate, often do not reveal important information about the models’ performance (e.g., spatial or temporal displacements; the reason behind a poor score) and are unable to accommodate these specific information needs. For example, summary statistics such as the correlation coefficient or the mean-squared error provide minimal information to developers, users, and decision makers regarding what is “right” and “wrong” with a model. New spatial and temporal-spatial tools from the field of weather forecast verification (where comparisons typically focus on much finer temporal and spatial scales) have been adapted to more completely answer some of the important earth system model evaluation questions. For example, these tools can be used to address spatial and temporal displacements in projections of El Nino-related precipitation and/or temperature anomalies. Examples of several applications of these tools in a climate context will be shown, using output of the CESM large ensemble. In general, these tools provide diagnostic information about model performance – accounting for spatial, temporal, and intensity differences – that cannot be achieved using traditional (scalar) model comparison approaches. Thus, they can provide more information that can be used in decision-making and planning. Future extensions and applications of these tools in a climate context will be described.