Verification of simulated radar reflectivity and echo-top forecast at NCEP

Simulated radar reflectivity and echo top are good indicators of detailed structure of convective clouds and precipitation forecast by mesoscale models, particularly as their resolutions have been increasing in recent years. Although these two products have been generated from NCEP's operational mesoscale models or systems for several years, their forecast performance has not been systematically evaluated nor compared. Recently we have dedicated much effort to do this over CONUS. This paper summarizes the recent verifications of these two products using hourly 1 km national 88-D radar mosaic data as 'truth'. The evaluated mesoscale models come from NCEP's operations and include 12km-NAM, 13km-RUC and 4+5km-WRF (NCEP-NMM + NCAR-ARW from HiResWindow), 32km-SREF (Short Range Ensemble Forecast System before and after its upgrade in 2009), and SREF in a downscaled 5 km form. The verification results show that (1) the reflectivity forecast has systematically better performance than that of echo top forecast in all single models, and has better performance for a weak dBZ threshold than for a strong dBZ threshold. Their equitable threat score (ETS) is around 12~16%, which is acceptable although it is still much below that of precipitation forecasts from the same models; (2) the SREF reflectivity and echo top forecasts show better performance than that of the control member base models if appropriate ensemble probability thresholds are selected; (3) the newly upgraded SREF significantly raised reflectivity and echo-top forecast scores compared to the previous version before the upgrade; and (4) if the SREF reflectivity and echo top forecast are downscaled from 40km to 5km resolution, both ensemble and its base models show much better performances.