Precipitation forecasts generated by mesoscale models are often difficult to evaluate because traditional threat scores are quickly saturated by minor differences in the precipitation fields. Composite sample methods offer a simple way to collect and evaluate the probability distribution functions of the modeled and observed fields for specifically defined events. False alarms and missed forecasts can be diagnosed using criteria based on average rain amount and intensity across the sample area. Other diagnostics can be derived regarding the structure of the PDFs given that an event is either predicted or observed.

The composite method has been applied to the operational COAMPS(tm) forecasts on the 27 km grid for the winter and summer precipitation regimes over the United States for 2002 and 2003. The initial results indicate that about 50% of the events sampled were false alarms or missed forecasts. The remaining 50% were relatively “good” forecasts in that the predicted and observed PDFs had similar structures. For these good forecasts, variability within the sample on any given day was quite high. But in general a forecast of precipitation meeting the event criteria would have been correct within the sample collection area on these days.