Applying a single-observation-forecast neighbourhood framework to the verification of km-scale NWP

Routine verification of deterministic Numerical Weather Prediction (NWP) forecasts from the convection-permitting and near-convection-resolving NWP models has shown that it is hard to consistently prove that the higher resolution model is more skillful. Here the use of conventional metrics and precise matching of the forecast to conventional synoptic observations in space and time, is replaced with the use of inherently probabilistic metrics such as the Brier Score, Ranked Probability and Continuous Ranked Probability Scores. These are applied to both single forecast grid points and forecast neighbourhoods.

Six surface parameters were considered: 2 m temperature, 10 m wind speed, total cloud amount (TCA), cloud base height (CBH), visibility and hourly precipitation. The results show that more than precipitation forecast skill is compromised when using a traditional verification approach. Adopting this inherently probabilistic approach enables the comparison of near-convection-resolving ensemble prediction systems (EPS). Thus far the strategy has been tested when comparing two models of different resolutions, and comparing deterministic to a convective-scale ensemble. Asm part of the "proof of concept" it has also been applied to the "test vs control" case which is so important for operational NWP in accepting model upgrade packages. The strategy also offers pointers for the optimization of post-processing to ensure optimal skill of forecast products.