Extended Abstract (832K)9.4
Observing verification trends and applying a methodology to probabilistic precipitation forecasts at a National Weather Service forecast office
Sam Lashley, NOAA/NWSFO, Syracuse, IN; and A. Lammers, L. Fisher, R. Simpson, J. Taylor, S. Weisser, and J. Logsdon
National Weather Service (NWS) Forecast Offices issue forecasts of probability of precipitation (PoP) in 12 hour intervals. These forecasts are issued routinely twice per day and cover a 7 day forecast period. Official verification of these 12 hour periods is completed by the NWS Office of Climate Water and Weather Services (OCWWS) Performance Branch using the Brier score method. Brier score is computed for NWS forecasts using the point forecast matrices (PFM) issued for points (specific locations) within the County Warning Forecast Area (CWFA) of local NWS forecast offices. Brier score is also computed for the same locations using data from the Model Output Statistics (MOS). NWS PoP forecasts are then compared to MOS forecasts using the Brier score.
While this is a commonly accepted method of verifying probability forecasts, it does not take into account the variability of forecasts from one run to the next. For example, it will be shown that MOS PoP forecasts can fluctuate considerably from run to run through a 7 day forecast, but still have good Brier scores. NWS forecasts can often follow MOS forecasts, which may lead to inconsistent forecasts through time. NWS customers may view this inconsistency as uncertainty, which may lead to a lack of confidence in precipitation forecasts by these users.
Forecasters at the NWS Northern Indiana office (KIWX) are cognizant of the uncertainty changing probabilities can have for local customers, especially those looking at specific forecast periods over several days. KIWX meteorologists employ a methodology of trying to be more consistent with their forecasts over time, trending from a lower probability forecast in day 7 to a high probability forecast in day 1 as confidence increases. This method reduces the variability of PoP with time and appears to NWS customers as a smooth, confidence building forecast. This method is also consistent with probability based forecasting methodologies.
This paper will look at 12 hour PoP forecasts, their trends and verification over 7 day forecast periods. We will look at Brier score computation as well as a relatively new verification method, known as the Ruth-Glahn Forecast Convergence Score (FCS). The FCS tracks a 12 hour forecast PoP through a 7 day forecast cycle and measures the number of significant changes from run to run for each forecast period. It will be shown that this method reflects a more realistic verification of how well PoP forecasts trend over time. It will also be shown how this forecast philosophy may help improve gridded verification in the NWS National Digital Forecast Database era as well as improve NDFD collaboration and consistency between NWS forecast offices.
Session 9, Forecast Evaluation II
Wednesday, 23 January 2008, 4:00 PM-5:00 PM, 219
Previous paper
Recorded presentation