A Comparison of QPF from 4 km Grid Spacing WRF Simulations with Operational NAM and GFS Output Using Multiple Verification Methods

The ARW-WRF model was run over the period from March through November 2013 with 4 km grid spacing to better understand the limits of predictability of short-term (12 h) quantitative precipitation forecasts (QPF) that might be used in hydrology models. Two sets of runs were performed, with one initialized using NAM and the other GFS output as the first guess field in the ARPS3DVAR system. Radar data were assimilated in both runs. Several verification methods were used to compare the QPF from these two high resolution runs with coarser operational GFS and NAM QPF. NCEP Stage IV precipitation data were used to represent ground truth in the verification process. Since traditional verification methods are sensitive to small-scale variability and provide limited information useful to model developers because they only evaluate grid-to-grid comparisons, two spatial verification techniques, neighborhood and object-base, were used to verify QPF for 1h, 3h, 6h and 12h precipitation accumulation intervals in order to provide information about what time and space scales may provide enough accuracy in the QPF for reasonable use in hydrology models. In addition, verification also examined location displacement, and intensity and structure errors. In general, as indicated by Fractional Skill Score (FSS), WRF showed a large advantage over NAM and GFS, especially for large thresholds, and also, the FSS of WRF increased noticeably with smoothing of scales. Of note, GFS performed much better compared to NAM, likely at least partly because the GFS simulations were coarser, and therefore were smoother after regridding to a higher resolution. The Method for Object-based Diagnostic Evaluation (MODE) was utilized to quantify location, intensity and structure errors between model simulations and observation. The frequency distribution of intensity sum difference between matched pairs showed WRF and GFS were generally too wet and NAM was the only model that did not have this bias, and may have even been a bit dry. As an encouraging sign of relatively small errors, the WRF showed a very high frequency count concentrated in small intensity sum difference bins. However, for several cases, WRF produced too large intensity errors, suggesting a need for further investigation of these “wet” cases. Diurnal cycles of verification skills between various hydrology models, as well as real-time weather analysis, were studied through these cases in order to acquire more comprehensive understandings of multiple verification methods. Case study analysis will be performed and presented at the conference for some of these outlier events.