The techniques are compared in two ways. First, the variability of the training models is compared by developing a number of subsets of the training data to develop multiple statistical models. The second comparison involves bootstrapping contingency statistics of the test cases, for comparison of the average and median statistics and confidence intervals to determine if one method is statistically significantly better than the alternative approach. Preliminary results indicate that the areal coverage and principal component analysis techniques perform similarly, suggesting the simpler, computationally less demanding, and easier to interpret areal coverage technique may be preferred for distinguishing major severe weather outbreaks from less significant events.
Additionally, the National Centers for Environmental Prediction (NCEP)/National Center for Atmospheric Research (NCAR) reanalysis (NNRP) dataset and the North American Regional Reanalysis (NARR) datasets are compared using the principal component analysis technique. The purpose is to determine if a higher-resolution dataset (such as the NARR) provides any additional information to distinguish outbreak types at their valid times. Preliminary findings suggest that reanalysis datasets with smaller grid spacing do not necessarily provide additional useful information. Reasons for these findings and implications on future studies will be discussed.