Application of Multi-Dimensional Stratification in Forecast Verification

Murphy (1995) extended the general statistical framework for forecast verification to include stratification of results by an external parameter. In general, statistics can be grouped or stratified into subsets by conditioning on another variable or parameter. In the case of forecast verification, selecting a parameter that is connected to the forecasting process and assessing the conditional distributions/probabilities of the statistics allows for additional insights into the performance of the forecast system. In this work, stratification of verification information was extended to multiple dimensions using multiple covariates. Decomposing forecast evaluation information in this manner allows for identification of situations where the forecast system provides exceptional performance as well as the situations where the forecasts underperform. This could provide useful guidance to the users of the forecast system as well as allowing developers to focus their efforts on improving the system in specific situations. At the conference, results from this approach will be demonstrated using examples from operational numerical weather prediction output and machine-learning generated probabilistic forecasts.