Using Data to Detect and Resolve Model Structural Errors [INVITED]

Short Abstract: In environmental studies, field data are typically used to evaluate model performance, estimate parameters, establish ‘credibility', select between models, or estimate prediction uncertainties. But the most important scientific use of data is to detect, diagnose and correct inadequacies in prior model structural hypotheses. This talk will discuss some ways in which data has been (can be) used to correct and improve existing model hypotheses, including methods to characterize informational content of data under uncertainty. Our goal is to promote discussion of how modeling can proceed under the admittedly challenging conditions where available data are uncertain, only partially informative, and not provide the basis for a complete test of hypothesis.

Longer Abstract: In environmental modeling studies, field data are typically used mainly to evaluate model performance, estimate model parameters, establish ‘credibility' of a prior model selection, select between alternative model structures, and/or to estimate model prediction uncertainties. But in contrast with the degree of effort expended, improvements in model reliability have (over the past several decades) been relatively small. In our view, most modeling studies do not target what is arguably the most important scientific use of data -- to detect, diagnose and correct inadequacies in prior model structural hypotheses, and to assist in the construction of improved models. Instead, there has been a focus on evaluating prediction uncertainty, and the power to discriminate between alternative model hypotheses remains so weak that many people now prefer to talk about multiple ‘equally likely' models.

In this talk we discuss some of the ways in which data have been (or can be) used to correct and improving existing model hypotheses. This clearly involves methods to characterize, quantify and/or evaluate the informational content and value of data, and must usually be done in a situation of considerable data uncertainty and error. Further, for modern very high order and/or complex environmental models, the available data may be only partially informative about certain aspects of the model, and can therefore not provide a sufficient basis for a complete test of hypothesis.

Our goal is to promote discussion of how the science of environmental modeling can proceed in a productive direction under these admittedly challenging conditions. To be successful, this discussion must involve the active participation of process scientists, modelers and systems theorists alike.