The Utility of an Ensemble Sensitivity-Based Subsetting Technique to Improve Mesocyclone Intensity Forecasts in the NOAA Warn-on-Forecast System

Ensemble sensitivity analysis (ESA) is a computationally inexpensive tool that identifies dynamics relevant to meteorological outcomes by using a linear regression to relate a forecast to the prior model state. This technique has been the subject of increased study recently due to its utility in operational forecasting and research applications. Because of its linear assumption, ESA has been most widely used with large-scale atmospheric dynamics as conventional belief has suggested that processes at such scales are often acceptably characterized as linear. However, study has recently extended to ESA’s potential in short-term mesoscale applications that are often believed to be highly chaotic—like convective processes—and results have proven generally promising that the technique can still find success with these phenomena. Left largely unaddressed is the potential for ESA utility with very small convective scales like those for individual thunderstorms. Fortuitously, the NOAA Warn-on-Forecast System (WoFS) is currently being tested to produce accurate, short-term forecasts of ongoing or imminent storms. Because WoFS aims to serve as the first large operational ensemble paradigm for short-term severe storm forecasts, this technique could theoretically prove useful in an eventual effort to improve warning lead times. Given that ESA is not designed to diagnose forecast sensitivities arising from differing model physics, it is not clear how ESA validity will interact with WoFS’s variation in parameterization schemes across members. Further, recent work has studied ensemble subsetting, a novel ESA-based tool which removes ensemble members with large errors in sensitive regions to form smaller, theoretically more accurate ensemble subsets. If negating factors for ESA utility in WoFS can be overcome, it is plausible that this method yields potential for automated forecast improvement of short-term convective signals in WoFS.

This study diagnoses the utility of ESA and its derivative tools in WoFS. Using a 36-member WoFS-imitating ensemble, updraft helicity (UH) sensitivity fields are calculated and qualitatively analyzed for a supercell case in May 2019 to determine a) the efficacy of ESA in WoFS, b) how the technique specifically interacts with the WoFS configuration, and c) dynamics relevant to mesocyclone intensity in a given scenario. The ability of ensemble subsetting to yield forecast benefit was then tested via two sets of experiments. The first attempts subsetting in an idealized setting in which a “truth member” is removed and compared to the subsets, serving as a theoretical upper limit to the utility of subsetting. A practical subsetting experiment was then performed using near-storm observations from the Targeted Observations by Radars and UAS of Supercells (TORUS) project.

Results suggest that ESA’s efficacy is systematically affected by its multiphysics nature such that state spread is stratified by boundary layer parameterization scheme. Idealized subsetting experiments show improvement in forecasts of mesocyclone intensity, with distance thresholding permutations having the most impact on skill. Practical subsetting results were less clearly beneficial with some indication of improved forecast skill depending on subset size, but verification methods and observation error raise questions about the trustworthiness of practical subset results. These analyses suggest that, despite challenges, ESA and its subsetting technique may represent worthwhile pursuits in endeavors to improve WoFS skill.