Assessment and Validation of an Observing Systems Simulation Experiment (OSSE) System Using a Summary Assessment Metric (SAM) Inter-comparison of OSSE and Observing System Experiment (OSE) Results

Observing systems simulation experiments (OSSEs) are used to simulate and assess the impacts of new observing systems planned for the future or the impacts of adopting new techniques for exploiting data or for forecasting. This study focuses on impacts of satellite data on global numerical weather prediction (NWP) systems. Since OSSEs are based on simulations of nature and observations, reliable results require that the OSSE system be validated. This validation involves cycles of assessment and calibration of the individual system components as well as the complete system, with the end goal of reproducing the behavior of real-data observing systems experiments (OSEs). This study investigates the accuracy of the calibration of an OSSE system before any explicit tuning has been performed by performing an inter-comparison of the OSSE summary assessment metrics (SAMs) with those obtained from parallel real data OSEs. The main conclusion reached in this study is that, based on the SAMs, the OSSE is able to reproduce aspects of the analysis and forecast performance of parallel OSEs despite the simplifications employed in the OSSEs.