Probabilistic Verification of Global and Mesoscale Ensemble Forecasts of Tropical Cyclogenesis

Probabilistic forecasts of tropical cyclogenesis have been evaluated for two samples: a near-homogeneous sample of ECMWF and Weather Research and Forecasting (WRF) Model–ensemble Kalman filter (EnKF) ensemble forecasts during the National Science Foundation's (NSF) Pre-Depression Investigation of Cloud-systems in the Tropics (PREDICT) field campaign (15 August–30 September 2010) and ECMWF ensemble forecasts during the 2010–12 Atlantic hurricane seasons. Quantitative criteria for tropical cyclone (TC) formation were first determined from model analyses based on threshold values of lower-tropospheric circulation, local thickness anomaly, and minimum sea level pressure. A binary verification was then performed for all ensemble forecasts with initial-time tropical disturbances. During the PREDICT period, the ECMWF and WRF–EnKF had similar verification statistics, with reliability diagrams of positive slope flatter than unity, and relative operating characteristic (ROC) curves that demonstrate skill. For the 2010–12 ECMWF ensemble forecasts, the equitable threat score was small and positive, with skill mostly lost after 5 days. The reliability diagrams for 1–5-day forecasts were monotonic increasing, though an overly large number of short-range ensemble forecasts predicted a low probability of a TC when a TC was verified. The ROC curves exhibited similar skill for forecasts out to 5 days. The reliability curves were sensitive to parameters such as time tolerance and threshold values, and insensitive to cases that originated from African easterly waves versus those that did not. Qualitative investigations revealed case-to-case variability in the probabilistic predictions. While the sample size was limited, the ensembles showed the potential for probabilistic prediction out to 5 days, though it appeared that the model struggled with developing a warm core in the short-range forecast.