Object-based Verification for the HRRR Model Using Simulated and Observed GOES Infrared Brightness Temperatures

In this study, the Method for Object-Based Diagnostic Evaluation (MODE) is used to assess the accuracy of the experimental High Resolution Rapid Refresh (HRRRx) model. The study focused on examining the characteristics of the upper level cloud field through comparison of the HRRRx forecast cloud objects to the observed cloud objects identified in Geostationary Observing Environmental Satellite (GOES) 10.7-µm infrared brightness temperature imagery. Two one-month periods were analyzed, 1-31 August 2015 and 1-31 January 2016, in order to assess the HRRRx forecast accuracy during both warm and cool seasons given potential differences in cloud characteristics. These differences include fewer cloud objects in 1-31 January 2015 compared to 1-31 August 2015 while the overall area of forecast objects is greater in 1-31 January 2015. These characteristics are consistent with the more predominant small-scale convective cloud features found during the summer compared to the larger synoptic-scale cloud systems more frequently observed during the winter.

The HRRRx accuracy is assessed using the newly defined MODE Skill Score (MSS). The MSS is an area-weighted calculation using the cloud feature match value from MODE that summarizes MODE output into a single number. Overall, it is found that the 1-h forecast is the most accurate for both months. Forecast accuracy was higher at shorter lead times during August, possibly due to better use of radar data by the data assimilation system during convectively active periods. Object attributes from each matched forecast-observed object pair for 1-31 August 2015 were individually analyzed to examine why the 1-h forecast is more accurate than the 0-h analysis. This analysis showed that while the displacement errors between the forecast and observed objects increases between FH0 and FH1, the size of the 1-h forecast objects better represent the observed cloud objects.