Benchmarking: An International Model Comparison

In recent years there have been many land surface model comparison studies covering a variety of spatial scales, climates and ecosystems. These studies have highlighted general weaknesses within the models and helped to set international development priorities. In essence, these studies have tried to answer the question “How good are land surface models?”, but have not attempted to address the question “Are our models good enough?”.

Establishing the definition of “good enough” is not obvious, but can ensure that any model development strategy is prioritised to deliver the maximum impact on model performance, with respect to the pre-defined criteria. This is the concept of benchmarking. Whilst definitive community benchmarks remain an unresolved research question, it is possible to establish minimum criteria that a land surface model should be able to achieve.

In an attempt to introduce the concept of benchmarking within an international comparison experiment, some simple benchmarks (such as a linear regression statistical model and a simple Penman-Monteith physical model) will be used to assess a number of land surface models. These models will be evaluated with data from a number of observational sites, representing various climatological environments.

The Protocol for the Analysis of Land Surface (PALS) models is an online tool for enabling such an analysis and will be used for this study. Results from the comparison will be presented and the performance of the land surface models relative to the simple benchmarks will be discussed. Strengths and weaknesses in the land surface models will be highlighted along with the main features of the benchmarks. These will be used to identify prioritised areas of development for land surface models.