Extensive interactions with water resources and watershed managers have revealed that decision makers have varying perspectives about climatic variability and opportunities for using climate forecasts to inform resource management decisions. In many cases, uncertainty about the accuracy of climate forecasts presents a formidable barrier to more effective use of forecast products. In addition, improper interpretation of climate forecasts can impede effective use of even the highest quality forecasts.

In response, we developed a forecast evaluation framework that provides consistency in assessing different forecast products, in ways that that allow individuals to use results at the level they are capable of understanding, while offering opportunity for shifting to more sophisticated criteria. Feedback from water and watershed managers confirmed the appropriateness of framework components.

We have incorporated the framework in an on-line forecast evaluation tool that users can customize to consider the regions, lead times, seasons, and criteria relevant to their specific decision making situations. It is presently implemented to evaluate the seasonal temperature and precipitation outlooks issued by the National Weather Service Climate Prediction Center. The tool also allows users to test their forecast interpretation skills, efficiently monitor the time evolution of the climate forecasts and subsequent observations, and place the forecasts in the context of recent and historical observations.

Ongoing forecast evaluations can counter poor perceptions of climate forecasts that derive from misperceptions about the forecasts and lack of demonstrated forecast quality in ways meaningful to potential users. Targeted evaluation of forecasts allows users to focus on results not clouded by forecasts they consider irrelevant; forecast evaluation results that are targeted for specific user requirements provide different assessments of forecast quality. The demonstrated skill, or lack thereof, provides a basis for exploring, with forecast users, forecast performance implications from a context of experience rather than conjecture. We have begun to work with forecast users to determine essential forecast attributes; requisite performance thresholds; relationships among forecast quality, utility, and value; and the potential utility and value of forecast improvements.