J2.1
The role of user-relevant verification in linking societal benefits to prediction systems
Laurence J. Wilson, MSC, Dorval, QC, Canada; and B. Brown
Forecast verification – the process of assessing the quality of forecasts – serves as a natural link between prediction systems and forecast improvement, and between forecasts and decision making. In particular, verification has at least two important roles in connecting societal benefits to prediction systems. First, the choice of prediction variables and forecast attributes to be verified has significant impacts on the direction of improvement of prediction models and forecasts. Second, meaningful information about forecast performance is required by users (or users' decision-making systems) for optimal decision making. Thus, user-relevant verification is a necessary (but often ignored or under-emphasized) component of prediction systems, and has a critical role in determining societal benefits of forecasts. Verification summaries which use traditional measures, for example those routinely prepared by operational centers, may be useful for some, but are typically widely disseminated as if they should satisfy all users of forecasts. Generally, the more care that is taken to design a verification system to meet the specific needs of a particular user group, the more likely it is that the verification system will fulfill its proper role as a link between prediction systems and their societal benefits. A number of new verification methods have recently been developed which go at least part way towards fulfilling the link between forecast systems and societal benefits.
This paper will include a survey of verification methods that provide user-relevant information and will consider limitations of current practices in model verification. Approaches for incorporation of user-relevant verification approaches in prediction system development and assessment, as well as societal and economic research linking to prediction systems will be discussed.
Recorded presentationJoint Session 2, Operational Perspectives on the Societal--Prediction Link
Wednesday, 23 January 2008, 10:30 AM-12:00 PM, 214
Next paper