A neighborhood-based probability of precipitation forecasting technique: Further testing

A neighborhood-based post-processing technique using a relationship between quantitative precipitation forecasts (QPF) and probability of precipitation (POP) forecasts was further evaluated to better understand its skill (measured by the Brier score and Brier Skill score) relative to other methods, and to determine if modifications could further improve skill. Most testing used ensemble data provided by the Center for Analysis and Prediction of Storms (CAPS) from the National Oceanic and Atmospheric Administration (NOAA) Hazardous Weather Testbed (HWT) spring experiments in 2007, 2008, and 2010 and focused on 20 km and 4 km horizontal grid spacings. Some additional work also made use of NAM QPF from 2009 and 2010.

Several tests were performed using only the CAPS output from the three years. In one test, different combinations of training and testing datasets using a single model member were evaluated using 6 hour accumulation periods. This test showed that the biggest improvements to POP skill happened when the technique was tested with the most recent and most skillful QPF. In addition, no significant change s in skill happened when training only on the 6 hour period of interest instead of all 6 hour periods within a 30 hour forecast. A second test that divided the model domain into four equal sub-regions to allow training and testing over smaller regions with more similar climatology and geography showed some potential for improvement in skill when using the smaller regions. A third test investigated t he sensitivity of skill to idealized changes in model QPF accuracy. Taking into account improvements in model accuracy, the neighborhood approach proved more skillful than a calibrated traditional approach normally applied to ensembles for small improvements in model forecast accuracy, but traditional approaches were more skillful when forecast accuracy improved further. When QPF skill was worsened, the neighborhood approach appeared to have an advantage over traditional approaches, suggesting its performance might beat that of traditional ensemble-based approaches at longer lead times.

Finally, to understand how the skill of the approach changed at lead times longer than the 30 hours available from the CAPS dataset, NAM output was used out to 84 hours. Skill did not rapidly deteriorate with longer forecast lead time, suggesting the approach may be particularly useful to forecasters at longer lead times.