A Comparison of QPFs in Two Ensembles from the 2016 Spring Forecasting Experiment with and without Radar Data Assimilation

In previous NOAA/Hazardous Weather Testbed Spring Forecasting Experiments (SFEs) as well as the 2016 SFE, various parameters of model forecasts were evaluated subjectively by participants and objectively using various metrics. During the 2016 SFE, one of the parameters that was examined was the impact of radar data assimilation on convection-allowing model (CAM) ensemble probabilistic radar reflectivity forecasts, a field that is strongly tied to precipitation. This study focuses on comparing quantitative precipitation forecasts QPFs from two different 10 member CAM ensembles from the 2016 SFE: one using radar data assimilation provided by the Center of Analysis and Prediction of Storms (CAPS) at the University of Oklahoma and another that did not use radar data assimilation provided by the National Severe Storms Laboratory (NSSL). Both ensembles had the same initial conditions, lateral boundary conditions, and physics, with the only difference being that the CAPS ensemble had WSR-88D radar data assimilated into the initial conditions while the NSSL ensemble did not. Two main goals of this study were: 1) to see how far into the 36 h forecast period the positive effects of radar data assimilation extended, and 2) to compare the subjective evaluations of the probabilistic reflectivity forecasts completed by the 2016 SFE participants to the objective evaluation metrics based on the QPFs from the CAPS and NSSL ensembles. Equitable threat scores (ETSs) were computed for precipitation thresholds ranging from 0.10 to 0.75 in. for each CAPS and NSSL member, as well as ensemble means, for 3-h accumulation periods. In addition, neighborhood-based ETS (ETS_r), which allows grid points within a certain radius to be considered when computing contingency table elements, was computed for radii ranging from 8 km to 60 km. The ETS difference between CAPS and NSSL peaked between hours 3 and 12, likely due to the benefits of radar data assimilation in those early hours. As the radius in the ETS_r increased, the difference in ETS_r between the CAPS and NSSL ensembles increased all the way out to hour 36. 3-h probabilistic QPFs were evaluated using the area under the ROC curve (ROC Area). The CAPS had ROC Area values of around 0.05 (or more) higher than the NSSL from hours 3 to 12 at the 0.10 and 0.35 in. thresholds, and from hours 3 to 15 at the 0.50 and 0.75 in. thresholds. Additionally, the subjective evaluations from participants of the 2016 SFE that answered the question: “How long into the forecast does the assimilation of radar data have a positive impact on the ensemble?” matched up fairly well with the hours that were statistically significant as per ETS and ROC Area differences.