An evaluation of impacts of grid resolution on the verification of aviation weather forecasts

Over the past several years the horizontal resolution of many operational numerical weather prediction models has increased a great deal. For example, the Rapid Update Cycle (RUC) has increased from 60-km resolution in the 1994-1998 period, to 40 km in the 1998-2001 period, to 20 km in the 2001-2005 period; and it will be increased to 13-km resolution sometime in 2005. As a result, many of the aviation weather algorithms applied to the model output have also increased in resolution. This study investigates the effects of the changing resolution of the RUC model on the standard verification statistics computed in evaluations of various aviation weather algorithms. Specifically, the impacts on verification statistics for algorithms designed to diagnose or forecast three different aviation weather phenomena (icing, turbulence, and ceiling and visibility) are evaluated. Pilot reports (PIREPs) of icing and turbulence are used as observations for the icing and turbulence verification analyses, and METARs are used for the evaluation of ceiling and visibility. Standard verification statistics, such as probability of detection of “YES” observations (PODy), probability of detecting “NO” observations (PODn), percent of the total airspace that is impacted by the forecast (%VOL), and area under the Relative Operating Characteristic (ROC) curve, are examined at different resolutions. In addition to grid resolution impacts, four different approaches for matching the forecast grids and point observation value are also investigated. These approaches include (1) using the most extreme value among the four gridpoints surrounding the observation; (2) using the average; and (3, 4) using two different types of interpolation. The evaluation includes several continuous months of forecasts and observations for each algorithm. Basic results indicate that many verification statistics are relatively insensitive to both the grid resolution and the matching method. In many cases, a change in resolution is manifested as a change in calibration in the ROC curve.