Adaptation of an Object-Based Verification Method for Moist Convection

Object-based verification offers an alternative to traditional verification metrics, motivated by deficiencies shown to exist when traditional metrics are used to evaluate convective-allowing limited-area models. One such object-based method, Structure Amplitude Location (SAL), was originally developed to evaluate forecast accumulated-precipitation fields, and this study describes a small modification for use with instantaneous composite reflectivity forecasts (where objects’ minimum size and reflectivity thresholds are prescribed). Both the original and modified SAL methods are used to evaluate daily 12-km North American Model (NAM) forecasts against NCEP/EMC 4-km Stage IV-accumulated precipitation estimates, during the summer of 2015 for a central US domain.

Results show substantial sensitivity to the reflectivity threshold. This is likely related to sampling more signals from convective cell cores, and progressively ignoring stratiform rain areas, as threshold increases. The primary difference between the two methods is a larger structural error in SAL using reflectivity, likely related to the unresolved convective peaks in the 12-km NAM forecasts; this error is smoothed out when accumulated precipitation is evaluated. SAL using reflectivity also reveals a diurnal cycle of skill, with minimum skill occurring around early to late afternoon local time, before average convective activity reaches its maximum; maximum skill occurs just before sunrise. Finally, lag of convective activity after the peak in SAL error suggests that development of convection may be responsible for much of the forecast error.