Forecast verification of extremes: Use of extreme value theory

Evaluating the ability of a weather forecasting system to predict extremes should be an important consideration in forecast verification, particularly given the well known societal impacts of extreme events. Yet statistical methods devised for extreme values have rarely, if ever, been applied. In the distributions-oriented approach to the verification process, the extreme levels of the weather variable being forecast would naturally be taken into account. But the common approach of forming a two-way contingency table for the joint distribution of forecasts and observations is of limited utility, because of sparse entries for extreme classes. Alternatively, this joint distribution could be represented by some parametric form. Yet, in practice, it is unclear whether any specific parametric form can adequately represent the behavior in the extreme tails. In the present talk, the proposed remedy involves adapting a method from the statistical theory of extreme values. This well-developed theory provides a family of parametric distributions with flexible tail behavior.

Making use of the calibration-refinement factorization, the focus is on modeling the extreme tails of the conditional distribution of the weather observation given a forecast. In this way, only the univariate (not the more complex bivariate) statistical extreme value theory need be considered. Extremes have two basic features, their rate of occurrence and their intensity. The exceedance of a high (or falling below a low) threshold by the weather variable is modeled by a conditional Poisson distribution, whose rate parameter is expanded as a function of the forecast. Likewise, the excess over a high (or deficit below a low) threshold of the weather variable is modeled by a conditional generalized Pareto distribution, whose scale parameter depends on the forecast. In this way, whether or not there is any skill (or how much skill exists) in forecasting weather extremes corresponds to determining whether or not (or to what extent) using the forecast as a covariate improves the fit (e.g., via a likelihood ratio test).

As a check, the proposed method is applied to Monte Carlo simulations for the hypothetical situation in which the joint distribution of forecasts and observations is a bivariate normal. It is further applied to a set of specialized NWS minimum temperature forecasts and observations at Yakima, WA. Because of the risk of damage to fruit buds from freezing during their development, these forecasts were made available to orchardists each spring.