A neighborhood-based probability of precipitation forecasting technique: Further testing
Several tests were performed using only the CAPS output from the three years. In one test, different combinations of training and testing datasets using a single model member were evaluated using 6 hour accumulation periods. This test showed that the biggest improvements to POP skill happened when the technique was tested with the most recent and most skillful QPF. In addition, no significant change s in skill happened when training only on the 6 hour period of interest instead of all 6 hour periods within a 30 hour forecast. A second test that divided the model domain into four equal sub-regions to allow training and testing over smaller regions with more similar climatology and geography showed some potential for improvement in skill when using the smaller regions. A third test investigated t he sensitivity of skill to idealized changes in model QPF accuracy. Taking into account improvements in model accuracy, the neighborhood approach proved more skillful than a calibrated traditional approach normally applied to ensembles for small improvements in model forecast accuracy, but traditional approaches were more skillful when forecast accuracy improved further. When QPF skill was worsened, the neighborhood approach appeared to have an advantage over traditional approaches, suggesting its performance might beat that of traditional ensemble-based approaches at longer lead times.
Finally, to understand how the skill of the approach changed at lead times longer than the 30 hours available from the CAPS dataset, NAM output was used out to 84 hours. Skill did not rapidly deteriorate with longer forecast lead time, suggesting the approach may be particularly useful to forecasters at longer lead times.