A New Object-Based Method for the Scale-Dependent Verification of Convection-Allowing NWP Models: Methodology and Application for the OU MAP Ensemble

A new object-based method consisting of an object trajectory (OT, 3d space-time object) identification algorithm and an OT matching algorithm is proposed for the verification of convection allowing NWP models. The new method is designed to (a) unambiguously and distinctly describe the high-resolution space-time structure of each individual convective storm with OTs and (b) match forecast and observed OTs that share similarities not only in the spatial patterns at certain time steps, but also in the temporal evolution of storms.

The method is applied to simulated reflectivity of a total of 27 10-member convection allowing ensemble forecasts generated by OU MAP (University of Oklahoma Multiscale data Assimilation and Predictability) lab during the 2018 Hazardous Weather Testbed Program Spring Forecasting Experiment. Forecast performance is evaluated via the following new aspects:

(a) Storm attributes such as duration, size, timing of convection initiation and dissipation are evaluated via sampling distribution of OT attributes. MAP ensemble captures both the distributions of OT size and duration and the diurnal cycle of convection initiation and dissipation quite well.

(b) An OT-based characteristic storm length (L) is derived with the purpose of estimating the strength of large-scale forcing for a certain period of time. The MAP ensemble shows good ability to distinguish between cases with significant convection and those with weak convection, as shown by high correlation between L(obs) and L(fcst) throughout the forecast period.

(c) Upon matching, timing and location errors associated with convection initiation are explicitly derived and evaluated. Scale dependency is explored by evaluating biases separately for short- and long-lived OTs. Statistically significant late (early) CI biases for convections initiated during the 12-18 (0-6 and 18-30) forecast hours are identified. The bias is more severe for short-lived storms (<6h) than long-lived storms. No significant CI location bias is detected.

(d) Parameters derived from OT matching are utilized to construct “optimally deformed” forecasts in which displacement errors are minimized. Novel insights on the scale dependency of ensemble skills are provided by evaluating the optimally deformed forecasts with Fraction Skill Score.