Skill scores associated with predictions of warm season convective system rainfall are generally far lower than those associated with cold season precipitation. Poor forecasting skill is related to the small-scale nature of warm season rainfall events and tight gradients of rainfall in space and time. Current skill measures are generally designed to do grid point to grid point comparisons. A model forecast showing a region of intense rainfall of a given complex shape will usually be considered to have no skill even if the simulation reproduced the complex shape exactly but with a small spatial error.

We have used a sample of around 60 6-hour periods of active convective system rainfall over the Upper Midwest simulated with a 10 km version of the Eta model to investigate both the impact of the verification grid box size on equitable threat score, and the changes in equitable threat score occurring when a phase-shifting approach is used to account for small spatial errors. The Eta model was run using both the Betts-Miller-Janjic (BMJ) and Kain-Fritsch (KF) convective schemes, with several modifications to improve the initialization of mesoscale features. In all, up to 14 model variants were examined.

Our results show that equitable threat scores rise signficantly when the 10 km Eta model output is averaged onto a 30 km grid and verified on that grid instead of on the model's own 10 km grid. The increases in skill are often comparable to the largest average improvements resulting from modifications in the initial conditions to better represent mesoscale features. Interestingly, when cases were rerun with a 30 km version of the model, the equitable threat scores for the 30 km output were often higher for a given rainfall threshold than the 10 km output averaged onto 30 km.

The phase shift verification technique permits the predicted rainfall to be shifted in one direction up to a maximum number of grid points to create the highest threat score possible. We applied the scheme in both the x and y directions. The average shift for the BMJ runs was 0.6 grid points, and 1.1 for the KF runs, implying that the spatial errors occurring from use of the BMJ scheme might be less than those associated with the KF scheme. The phase-shifted BMJ scores also improved more than the KF scores, accentuating the differences already present in the unadjusted equitable threat scores.