NOAA testbeds such as the Hydrometeorology Testbed - West (HMT-West) have been exploring the use of high resolution ensembles for several years. The evaluation of the ensemble mean and probability fields are hindered by the same challenges faced by traditional evaluation of high resolution deterministic forecasts. The addition of comparing to operational baseline models with coarser resolution, such as the Global Forecast System (GFS) and Short Range Ensemble Forecast (SREF) system, adds to the challenge. Typically traditional scores tend to be lower for high resolution models due to the penalty incurred by the smaller grid spacing and hence greater likelihood of missing the forecast by a few grid-point. Packages such as the Method for Object-based Diagnostic Evaluation (MODE) were developed to address this evaluation issue from an object based approach. MODE is part of the Model Evaluation Tools (MET) software package developed at the Developmental Testbed Center (DTC).

The DTC used MET and MODE to objectively evaluate the HMT-West Ensemble during 2010 and 2011 winter seasons and compared it with operational baselines. Much of the DTC analysis-to-date has focused on traditional scores. Remarkably, this analysis indicates that event at thresholds greater than 12.7 mm (0.5 inches) over 6 and 24 hours the HMT-West Ensemble exhibited greater skill than the GFS and sometimes the SREF. This presentation will shift attention to the use of MODE objects for understanding why the traditional scores turn out so favorably for the HMT-Ensemble despite the odds. It will also explore the use of MODE in some less traditional ways, such as forming objects of the probabilistic fields and evaluating against the gridded Quantitative Precipitation Estimates (QPE fields) and using MODE attributes of individual members to form ensemble attributes for ensemble verification.