Multi-Year Comparisons of OU MAP Real-Time Convection Allowing Ensemble Forecasts Produced During 2017-2022 HWT Spring Forecasting Experiments

Multi-year Comparisons of OU MAP Real-Time Convection Allowing Ensemble Forecasts Produced During 2017-2022 HWT Spring Forecasting Experiments

Brett Castro, Nick Gasperoni, Xuguang Wang, Yongming Wang

ABSTRACT

The University of Oklahoma (OU) Multiscale Data Assimilation and Predictability (MAP) Laboratory has participated in producing real-time convection-allowing ensemble forecasts during past Hazardous Weather Testbed (HWT) Spring Forecasting Experiments (SFEs) from 2017-2019 and 2021-2022. During these years, novel data assimilation (DA) methods were examined on improving the accuracy and prediction of convective system evolution across the continental US. In particular, the system includes the direct assimilation of radar reflectivity within the Global Statistical Interpolation (GSI)-based ensemble-variational (EnVar) DA scheme. Most recently, in 2021 and 2022, the OU MAP system was designed to mimic the approach of the future Rapid Refresh Forecasting System (RRFS) within the Unified Forecast System (UFS) effort. This design includes coupling EnVar with the Finite Volume Cubed Sphere (FV3) Limited Area Model (FV3-LAM) and hourly multiscale DA of mesoscale in situ and convective-scale radar reflectivity observations. During each SFE, the OU MAP system initialized 36-h ensemble free forecasts from the final DA analyses at 0000 UTC with ten ensemble members total.
The purpose of this study is to compare the forecasting performance across different years by comparing ensemble forecasts with observations across 10 high impact cases selected from each year. This talk will primarily focus on comparisons of 2021 and 2022 configurations to examine the improvements made between RRFS-like systems due to updates between each year. Multiple neighborhood-based methods are used to quantitatively examine accuracy of different aspects of the system. The standard neighborhood method is used, where binary probabilities of threshold exceedance are averaged within a predefined radius and among all ensemble members to produce neighborhood ensemble probabilities (NEPs). NEPs are produced for simulated composite reflectivity and 1-h precipitation at different thresholds and compared to observed neighborhood probabilities (NPs) produced by neighborhood averaging of MRMS observations. Additionally, a surrogate severe method is also incorporated to evaluate model predictions of severe storm characteristics, by converting simulated 24-h ensemble maximum updraft helicity (UH) tracks into “surrogate severe reports” and comparing them with observed SPC severe report locations of hail, wind, and tornadoes.
Promising trends in model accuracy were observed between the two years, with average skill scores in 2022 outperforming 2021 across most thresholds and hours in the next-day period (12-36). Results support improvement in both placement of convection and precipitation with the greatest improvements generally occurring at lower thresholds and larger neighborhood averaging radii. However, analysis of surrogate severe verifications were mixed, with 2022 showing greater consistency among selected cases but 2021 having higher skill among most larger UH thresholds and smoothing radii. Further results will compare the 2021 and 2022 performance to ensemble forecasts from 2017-2019 OU MAP systems, which can determine the relative performance of different model cores as previous years were coupled with NMMB and HRRR model configurations. Such a comparison will help to contextualize the progress and current status of RRFS-like ensemble CAM system in relation to previous year performances.