Extended Abstract (180K)11.3
Aligning forecast verification with user-specific needs - an example for aviation
Michael P. Kay, NOAA/ESRL/CIRES/Univ. of Colorado, Boulder, CO; and S. Madine, J. Mahoney, and J. E. Hart
Brier and Allen (1951) state that there are three primary purposes for performing objective evaluations of forecast quality: administrative, scientific, and economic. The separation of administrative purposes from scientific purposes is not particularly significant in practice. The primary difference is that administrative verification is closely linked to operational forecasting systems while verification of developing forecast systems relies, or should rely, more heavily on scientific, or diagnostic, approaches. In this broad view, the information needs of both the producers and consumers are not explicitly separated. Clearly, forecast producers have different needs from verification than consumers and therefore benefit from different verification information than forecast consumers. Modern forecasting systems are often comprised solely of computer algorithms rather than human-generated content. For these types of systems where algorithms can be changed and re-run on identical inputs, verification can guide the design and implementation of the algorithms and lead the producers to improve the product before it becomes operational. For the forecast consumer, the goal is no longer about improving the forecast as much as possible but rather about making the best decisions possible using the available information. The obvious difficulty with separating the problem into producer and consumer views is the clear identification of all distinct consumers and their particular needs for a forecasting system and the subsequent verification. Consumer-oriented verification can be seen as a cross-cutting approach that delves significantly into both the administrative and scientific applications of verification information targeted at a specific user or set of users with similar information needs.
This talk will focus on the results of a recent forecast intercomparison for several different convective forecasts used for making strategic decisions at the FAA Air Traffic Control System Command Center (ATCSCC). Results will be presented to contrast a purely administrative view of the verification information without a specific user to results when a specific user with known information needs is identified. The specific user needs of the ATCSCC also call for the transformation of the forecasts such that the definition of the event being forecast is common across a set of disparate forecasts. Details of this transformation, and the effects on the verification, will be presented.
Session 11, Verification
Thursday, 24 January 2008, 11:00 AM-12:15 PM, 226-227
Previous paper Next paper
Recorded presentation