During the development phase of an air quality dispersion model and in subsequent upgrades, model performance is constantly evaluated. These evaluations generally compare simulation results using simple methods that do not account for the for the fact that models only predict a portion of the variability seen in the observations. To fill a part of this void, the U.S. Environmental Protection Agency (EPA) developed a standard that has been adopted by the American Society for Testing and Materials (ASTM), designation D6589 - 00 for Statistical Evaluation of Atmospheric Dispersion Model Performance. Within the annex to this standard is an "example" test method that test the ability of dispersion models to simulate the average centerline concentration. The method involves grouping data into groups or regimes, in which the dispersion is expected to be somewhat similar. The average centerline concentration is then derived for each group using bootstrap resampling. It is this average centerline concentration that is then compared with the modeling results. By this means, the focus is on testing the ability of models to replicate the first moment (the average) of the centerline concentration distribution, which for most operational models is the only feature in the centerline concentration distribution they are capable of simulating.

This paper will focus on recent work to further test the ASTM "example" test method. This work involved the application of the test method to the results from ISCST3 (version 00101), AERMOD (version 01247), HPDM (version 4.3, level 920605), and ADMS 3.1. Results from three atmospheric dispersion field studies are analyzed - Prairie Grass (in 1956, rural, low level release), Kincaid (1980, rural, elevated release) and Indianapolis (1985, urban, elevated release).