Proactive QC based on Ensemble Forecast Sensitivity to Observations (EFSO) and Ensemble Forecast Sensitivity to observation error covariance matrix R (EFSR)

Despite recent major improvements in numerical weather prediction (NWP) systems, operational NWP forecasts occasionally suffer from an abrupt drop in forecast skill, a phenomenon called "forecast skill dropout." Recent studies have shown that the "dropouts" occur not because of the model's deficiencies but by the use of flawed observations that the operational quality control (QC) system failed to filter out. Thus, to minimize the occurrences of forecast skill dropouts, we need to detect and remove such flawed observations.

A diagnostic technique called Ensemble Forecast Sensitivity to Observations (EFSO) enables us to quantify how much each observation has improved or degraded the forecast. A recent study (Ota et al., 2013, Tellus A) has shown that it is possible to detect flawed observations that caused regional forecast skill dropouts by using EFSO with 24-hour lead time and that the forecast can be improved by not assimilating the detected observations.

Inspired by their success, in the first part of this study, we propose a new QC method, which we call Proactive QC (PQC), in which flawed observations are detected 6 hours after the analysis by EFSO and then the analysis and forecast are repeated without using the detected observations. This new QC technique is implemented and tested on a lower-resolution version of NCEP's operational global NWP system. The results we obtained are extremely promising; we have found that we can detect regional forecast skill dropouts and the flawed observations after only 6 hours from the analysis and that the rejection of the identified flawed observations indeed improves 24-hour forecasts.

In the second part, we show that the same approximation used in the derivation of EFSO can be used to formulate the forecast sensitivity to observation error covariance matrix R, which we call EFSR. We implement the EFSR diagnostics in both an idealized system and the quasi-operational NWP system and show that it can be used to tune the R matrix so that the utility of observations is improved.

Proactive QC, if implemented into the operational system, will allow us to build a database of defective observations. We believe that we can help instrument and algorithm developers to identify and fix potential flaws in their algorithms by providing such database along with relevant metadata.

Another important application of this study is to use EFSO and EFSR to accelerate the development of optimal assimilation methods for new observing systems. Current approach based on Observing System Experiment (OSE) has difficulties in obtaining statistically significant signals from new observing systems in the presence of the already abundant observations that are available in the "control" system. EFSO should address this issue by finding forecast impact of each observation and allowing the comparison of the impacts of different observation processing algorithms. Another non-trivial issue in designing assimilation methods for new observing systems is how to optimally specify the R matrix for them; our EFSR diagnostics can be used in this respect.