The Use of Large Ensembles in Climate Model Consistency Testing (Invited Presentation)

Constant evolution and improvement of large-scale climate simulation codes such as the Community Earth System Model (CESMÔ) necessitate quality checks to verify the constitution of new climate simulations. Large ensembles have proven particularly useful in this context, as they serve as a reference base for statistical consistency testing in situations where bit-for-bit reproducibility is not feasible. This is the case for virtually all situations of practical interest such as porting to a new computer architecture or environment, implementing minor code optimizations, or more substantial changes (that are not expected to affect the climate) such as testing alternative parameterizations. The CESM Ensemble Consistency Test (CESM-ECT) was developed for this purpose, utilizing a testing framework based on the popular technique of Principal Component Analysis to determine whether a set of new simulations is statistically distinguishable from an established ensemble of climate simulations. The test, which requires only data from the first nine model time steps, has proven very powerful and of high practical use and is now part of the official CESM release. We will present results from various case studies as well as our ongoing efforts to determine root causes for cases where the test indicates a statistical inconsistency.