Verification of forecasts is necessary in order to improve the quality of the forecasts. Much work has been done on developing objective verification techniques, methods, and scores to accommodate the diversity of forecasts and observations that are available in meteorology. In addition to objective methods for verification there exists a broader class of ideas and techniques which will be referred to as subjective methods that are useful for evaluation of forecasts. Subjective evaluation of forecasts is perhaps most useful for short-term areal forecasts such as those produced by the Aviation Weather Center and the Storm Prediction Center rather than point-specific forecasts.

There are many difficulties associated with collecting ancillary data about forecaster beliefs and concerns when forecasts are produced. Some of these difficulties include deciding which information to collect and developing efficient methods for the collection. Potentially, an organized system of data collection could be developed such that subjective information could be tied to objective verification measures. The unification of these two disparate but important approaches to verification can yield insight into forecast strengths and weaknesses based on stratifying the forecasts themselves. Perhaps the most natural stratification is that of meteorological conditions such as synoptic conditions however other stratifications are also possible (e.g., forecaster confidence in severe weather occurring or not).

The purpose of this paper is to outline some preliminary results of combining subjective and objective verification information using data from the 2001 National Severe Storms Laboratory/Storm Prediction Center Spring Experiment. Basic objective verification of the forecasts will be presented. Summary information will be presented on data collected from forecasters during the experiment and used to stratify the objective verification results. Discussion of the merits and limitations of such an approach as applied to this dataset will also be included. ~