Outside the Envelope: When Ensemble Forecasts Go Awry

Global ensembles are seeing increasing use among operational forecasters in the National Weather Service, both for creating public forecasts and communicating uncertainty to partners. Since ensembles entered operations, forecasters have indicated a tendency for events to verify "outside the envelope" (OTE) due to biases and/or underdispersion, leading to a perception that ensembles fail to represent the true realm of possibilities. However, there is limited information regarding the actual frequency of OTE events in current operational systems.

Five years (2008–2013) of 500-hPa geopotential height forecasts for the contiguous United States are examined to identify OTE events in four systems: the Global Ensemble Forecast System (GEFS), the Canadian Meteorological Center ensemble (CMC), the North American Ensemble Forecast System (NAEFS), and the European Center for Medium-Range Weather Forecasting ensemble (ECMWF). Results are presented for: 1) the frequency and spatial/seasonal patterns of OTE events, 2) the relationship between OTE frequency and lead time, and 3) trends in OTE events as ensembles improve.

Although OTE events typically cover a small area, there are occasional cases where the entire ensemble envelope is 180 degrees out of phase with the verifying analysis over a large portion of the study domain. Several of these notable cases are analyzed in detail to track the growth of errors and identify synoptic and mesoscale factors that may be associated with exceptionally poor forecasts.