The development of forecast confidence measures using NCEP ensembles and their real-time implementation within NWS web-based graphical forecasts

Ensemble model data can provide a wealth of knowledge to forecasters especially in terms of forecast confidence. A model run where members diverge correspond to a low confidence forecast while a model run where members converge correspond to a forecast of high confidence. The current NWS graphically based forecasts accessible to the public do not show this measure of uncertainty and thus communicate a false sense of accuracy. Starting in August 2004 and extending into 2006, analysis of about 3 years of individual GFS model ensemble data will occur by month and forecast length for each grid point. What will result is a climatology of each ensemble member, which will obviously not match the observed climatology based upon the NCEP reanalysis. The GFS model ensemble climatology will be normalized so that eventually there will be a mapping between the model ensemble value and the real-world value. Since there is a limited amount of data at this point, the climatology will be based on a 45 day running climatology. Variables include MSLP, geopotential heights, temperature, dewpoint, wind speed and precipitation among others. Once the normalized climatology distributions have been calculated for each grid point within the ensemble member, per month or season confidence measures will be developed from comparing the normalized spread of the ensemble members. This normalized spread will also be compared to the typical spread for that time of year and location to arrive at a relative measure of forecast uncertainty. If the current model uncertainty is greater (less) than the uncertainty of the model climatology, then there is a lower (higher) than average confidence. Preliminary confidence graphics have been developed and analysis to see how confidence values behave with certain synoptic situations is underway. It has also been seen that there is statistical significance between NWS forecast error during low confidence and high confidence regimes. Average NWS error for the below (above) normal GFS confidence forecasts was 5.32oF (3.46oF). A student t-test on these values revealed that there is a statistically significant difference to 95% confidence of the mean forecast error during low and high confidence GFS forecasts. That is, the mean WFO forecast error is significantly increased during times of low forecast confidence in the GFS ensemble. Therefore, forecasters have a-priori knowledge of the likely human forecast error when they see the GFS ensemble output-- before the NWS forecast even verifies. Although the confidence graphics are only based off of the GFS ensembles as of now, more models will be added to see how they behave when compared to each other. Recent feedback from NWS employees suggest an additional development of confidence graphics based on the “poor man's ensemble”. Eventually these graphics of below and above average confidence will be implemented into the Graphical Forecast Editor (GFE) for use in the National Digital Forecast Database (NDFD).