Using analysis uncertainty estimates from the Real-Time Mesoscale Analysis in the verification of grid-based forecasts

Evaluation of a grid-based forecast requires an objective analysis of observed data to serve as a “truth” grid. Forecasters in National Weather Service Western Region (NWS/WR) are concerned about their forecasts being evaluated against an objective analysis. For example, forecasters fear being penalized for adding detail to a valley with few or no observations because the analysis may not capture the mesoscale feature they are forecasting. In addition, forecasters are concerned about errors in the observations and background field influencing the analysis.

To support the evaluation of grid-based forecasts from the NWS National Digital Forecast Database (NDFD), the Analysis of Record project was started in 2004. The development of the Real-Time Mesoscale Analysis (RTMA) by the National Centers for Environmental Prediction/Environmental Modeling Center is the initial step in this project. The RTMA is performed in near-real time on the same 5-km grid as the NDFD. The developers of the RTMA have recognized the need for a grid conveying the uncertainty in the analyses. Experimental uncertainty grids became available in early 2007.

To mitigate the influence of analysis errors, grid-based forecasts are evaluated based on the magnitude of RTMA uncertainty estimates. Using a distributions-oriented approach to forecast verification, forecast errors will be binned based on how much they differ from the RTMA analysis uncertainty. Under this scheme, forecasts verifying within the bounds of the analysis uncertainty will not be penalized.

Examples of NDFD forecasts from mesoscale weather events over the western United States will be shown to demonstrate the verification approach. Results will be compared to traditional measured-oriented verification statistics to draw conclusions about the behavior of the technique.