Using an ensemble of nature runs to calibrate the impact from new observing platforms within meso- and convective-scale OSSEs

Convection-allowing model forecasts can skillfully predict the evolution of thunderstorms, but their skill is sensitive to errors in the initial environment. To improve representation of the environment, atmospheric observations are frequently assimilated. Gaps in observational coverage can reduce the effectiveness of data assimilation (DA) systems and allow error to persist in the environment. Many novel observing platforms are now becoming available that should improve the observational coverage of the atmosphere, but as yet have not been assimilated into numerical weather prediction models. Observing system simulation experiments (OSSEs) are frequently used to determine how to optimally assimilate these novel observations (e.g., spatial and temporal density) and understand their impact on forecast performance. OSSEs assimilate simulated observations extracted from a high-resolution simulation (i.e., a nature run) that closely resembles an observed weather phenomenon. Since simulated observations are created synthetically, the impact from various platforms and networks having different spatial and temporal characteristics can be tested.

While OSSEs are an effective tool to test the impact of different assimilated observation types, extensive testing is required to optimally tune each data assimilation parameter. To do this, many OSSEs and real-data experiments conduct data assimilation experiments using several configurations to determine which results in the most skilled forecasts. This approach helps to establish an adequate data assimilation configuration, but may not ultimately result in optimal system performance. Even if forecasts are initialized with accurate initial conditions, subtle errors in the environment and model physics often cause forecast skill to degrade with time. For instance, the evolution of convective storms is highly sensitive to modest changes in the environment. Many of these features occur at spatial and time-scales that are too small to be observed, and thus, remain a large source of forecast uncertainty. Importantly, the error growth is highly case and flow dependent (Melhauser and Zhang 2012).

Melhauser and Zhang (2012) outline two forms of predictability associated with forecast errors from a meso- and convective-scale OSSE experiment. When the flow has a dominant solution, OSSEs show that a reduction in analysis error via the inclusion of new observations leads to a reduction in forecast error. Alternatively, without a dominant solution, a reduction in the analysis error does not, on average, improve the forecast error. In practice, most OSSEs will have a mixture of these two types of predictability. The focus of this research is to measure and use the intrinsic predictability from a given OSSE case as a way to “calibrate” the results, i.e., to place any forecast error reduction in the context for the given experiment’s intrinsic predictability limit.

Typically OSSEs create a nature run and assume it to be the “true” solution (here we assume a perfect model). However, the use of a single deterministic nature run cannot measure the intrinsic predictability of the flow. The work here introduces the use of an ensemble of nature runs to provide insight into the intrinsic predictability by adding subtle changes in the environment to see how small forecast errors grow with time. The nature run ensemble then provides a range of likely outcomes for the evolution of individual convective cells. It also provides an envelope of likely outcomes to verify the OSSE forecasts against, and by using the error growth associated with the intrinsic predictability estimate, provides further insight into the forecast errors associated with the traditional OSSE experiments.

Figure 1 shows the results from an OSSE case testing the forecast impacts from radars and a network of profilers within a strong idealized quasi-linear convective system (QLCS) discussed in Labriola et al. (2023). The addition of the profiler network does improve the skill of the forecasts. The “skill” (e.g., the intrinsic predictability) of the ensemble of nature runs is shown in yellow, and suggests that further improvements using other profilers and/or networks may be limited given the intrinsic predictability. Here the estimate of the intrinsic predictability can help guide the design of the OSSE experiments.

This study will adapt this methodology onto a real-data case. It will use the initial and boundary conditions from NSSL’s Warn-on-Forecast System (Heinselman et al. 2024) for a spatially confined tornado outbreak that occurred in central Oklahoma on April 19, 2023. The initial ensemble should have more realistic errors than that reported by Labriola et al. (2023). The work will test radar and radar plus profilers to examine the impact of assimilating these new observations. The ensemble nature run will then calibrate the case’s intrinsic predictability and will also be used to verify the OSSE ensemble forecasts.

Heinselman, P., P. C. Burke, L. J. Wicker, and coauthors, 2023: Warn-on-Forecast System: From Vision to Reality. Submitted to Wea. Forecasting, Aug 2023.

Labriola, J., J. Gibbs, and L. J. Wicker, 2023: A method for generating a quasi-linear convective line suitable for observing system simulation experiments. Geosci. Model Dev., 16, 1779–1799

Melhauser, C. and Zhang, F. 2012: Practical and intrinsic predictability of severe and convective weather at the mesoscales. J. Atmos. Sci., 69, 3350–3371.