A Community Effort to Unify Verification and Validation Efforts

An evidence-based evaluation of all components of the Unified Forecast System (UFS) and the coupling between components is needed to make effective decisions guiding the development of the UFS and to ensure that the new systems are better than those being replaced. Ultimately, the goal is to develop unified verification approaches and common metrics across the user community to provide useful feedback to model developers. As a point of reference, verification relates to the weather prediction context of using observations to assess model forecast skill.

Under the auspices of the US Next Generation Global Prediction System (NGGPS) program, an endeavour has been undertaken to develop a comprehensive verification capability for the UFS. At the core of these activities is the UFS Verification and Validation Working Group (V&V WG). This group represents a wide cross-section of stakeholders and users and is actively endeavouring to define metrics and decision tools for evaluating the UFS. It has joined with the Developmental Testbed Center on gathering an initial set of metrics that are meaningful to most applications. This group is also examining ways to synthesize and interpret these metrics and develop process-oriented diagnostics. It is intended that the recommendations of the UFS V&V working group will also be used as guidance in the development of METplus. This presentation will highlight the UFS V&V discussions and some recommendations for using verification to inform evidence-based decision making. It will also highlight examples of how research to operations to research (R2O2R) is evolving through participation in the working group.