Object-based verification approaches for NWP

The authors describe object-based methods for verifying numerical forecasts. First, a recently developed method for defining rain areas for the purpose of verifying precipitation is described and examples presented from simulations using the Weather Research and Forecasting (WRF) model. This method reduces the high dimension of forecasts and observations by defining coherent rain areas, characterizing areas in terms of geometric properties and intensity, and matching forecast and observed areas so as to quantify biases in model behavior. The key utility of this object-based approach is defining measures of model performance that are diagnostic in that they convey potential sources of error in addition to the errors themselves. Major findings are systematic spatial errors for rain areas as a function of the diurnal cycle, a sharp dependence of forecast skill on the size of rain areas, and an overly narrow distribution of rainfall within areas likely due to cumulus parameterization.

The authors then describe a second object-based technique used to compare forecast and observed surface mesonet observations. This technique defines objects as sufficiently large temporal changes within time series at individual stations. This metric represents one of several metrics that quantify the skill as a function of increasing tolerance for temporal displacement of significant features. Using this approach they are able to show where and when forecasts using a finer grid increment are superior to forecasts with the same model (the Penn State/NCAR model, MM5) with a coarser grid increment (3.3 km versus 30 km). Furthermore, this approach yields valuable information about the predictability of terrain-induced motion.