Assessing systematic impacts of physics schemes in the NSSL Warn-on-Forecast System

Forecast inter-comparison studies of convection-allowing models (CAMs) are needed to better understand the impacts of model and ensemble configurations on storm forecasts and thereby optimize future operational CAM systems. We have recently developed novel, object-based analysis techniques to identify systematic inter-model differences in the representation of storms and near-storm environments. These techniques were initially applied to next-day forecasts from the Community Leveraged Unified Ensemble run during the 2017 NOAA Hazardous Weather Testbed Spring Forecasting Experiment (SFE; Potvin et al. 2019, in press). The analysis techniques have since been extended to 3-h forecasts generated by the NSSL Warn-on-Forecast System (WoFS) during the 2017-18 SFEs. The WoFS is a 3-km ensemble that assimilates radar and satellite data every 15 min (and conventional observations hourly) and issues forecasts every 30 min. The WoFS uses physics diversity, enabling comparison of forecasts whose models differ only in planetary boundary layer (PBL) or radiation scheme. The present work assesses systematic impacts of individual physics schemes by comparing composites of hundreds of storm forecasts generated using each of the six unique physics combinations in the WoFS.

Several key conclusions have already emerged from this new work. Systematic forecast differences between ensemble members using different PBL schemes are much larger than forecast differences arising from different radiation schemes. Verification with ASOS observations reveals the YSU PBL scheme produces much lower bias in temperature and dewpoint than the MYJ and MYNN schemes, which are too cool and moist. All three schemes are associated with equally skillful forecasts of storm location. The overall magnitude of PBL scheme impacts varies non-monotonically with forecast initialization time (19 UTC - 02 UTC in this study) and lead time (0-3 h). The YSU physics produces the least optimal storm environments and weakest storms of the three schemes, while the MYJ physics produces the most optimal storm environments and strongest storms. Inter-PBL forecast differences themselves vary both geographically and with the observed environment. Efforts are underway to compare observed and WoFS soundings to assess PBL scheme impacts on important severe weather parameters (e.g., MLCAPE, 0-3-km SRH) and vertical profiles of temperature, moisture, and wind.

The knowledge generated by this project will inform the development of ensemble forecast post-processing and calibration methods (e.g., ensemble subsetting and weighting) for both the WoFS and other physics-diverse CAM ensembles. In addition, the object-oriented analysis methods developed herein could facilitate forecast evaluations for other localized atmospheric phenomena, including tropical cyclones.