Adaptation of Advanced Weather Forecast Verification tools to Climate Problems: Opportunities and Challenges

Tools that measure the ability of a weather forecast to properly anticipate actual weather evolution have become much more creative and flexible than have climate model evaluations, which have somewhat stagnated at the visual map comparison level. One simple reason is that a weather forecast is hardly ever too far off the real weather in both time and space. Tools to describe the somewhat subtle differences, therefore, have to focus on a suite of nuances that can be extracted as indicators of mis-represented processes. A climate model, however, is never actually even expected to reproduce exactly the sequence of events the Earth System is undergoing. On one hand the time periods over which such comparisons are done are generally too short to sufficiently eliminate the contribution of internal climate variability. In fact, even the traditional 30-yr interval used for establishing climatologies might have already been demonstrated to be too short when considering decadal to multi-decadal prediction. On the other hand, the evaluation of climate models generally is done from the same meteorological and climatological perspectives as weather forecast models, using observables — fields easily measured — as reference. But the complexity of the system emerges when the output is used for decision making. Now it is no longer the mean temperature or the precipitation fields, their average variability, or the magnitude of extremes that are critical. An impact of the weather is expressing itself through flooding, dried up vegetation, all inherently complex and by their nature multi-variate with clear dependence on exact succession of events.

Slowly, conditional considerations and performance evaluations are making their way into the benchmarking and performance measurement domain of climate models. While model developers can certainly come up with a host of conditional links between fields, they often lack salience. Bringing in user perspectives, however, offers interesting and tested ways of identifying different ways to quantify the effects of good or poor performance across two or more variables. Users therefore indirectly know from their daily operations what the consequences are of a suite of conditional process links.

Here we use advanced weather forecast verification tools to capture the probabilistic nature of some climate problems. Object-based evaluations also provide good instruments to tackle different questions than what is traditionally done. When combined with conditional settings, then multi-variate context becomes an active part in the verification. In fact, some of the most integrated factors arise from such conditions, many of which might even remain unknown. Yet, they allow for more integrated insight into the processes that are to be captured and measured in their joint behavior. We demonstrate opportunities and challenges that emerged in the effort to adapt the weather forecast verification tools to climate problems when working with the water sector. It is investigated if not also new and creative tools could emerge for the climate domain.