East Coast vs. West Coast: A documentation of model forecast failures for Eta, NAM, GFS, GEM and ECMWF

In order to assess and compare the abilities of various forecasting models to predict high-impact weather events, it is helpful to document overall forecast performance and the occurrence of large forecast errors. This study documents such errors and compares the performance of five operational models along the United States East and West Coasts.

Model forecast errors in sea level pressure (SLP) that occur along the East and West Coasts are documented for 24-, 48-, and 72-hour forecasts. Observations from coastal buoys are compared directly to interpolated model forecasts from November through March for multiple years. SLP is selected for comparison since storm strength and location can be defined using SLP. The European Centre for Medium-Range Weather Forecasting (ECMWF) model, Global Forecasting System (GFS), Eta (through its discontinuation), North American Mesoscale (NAM) (from its inception), and the Canadian Meteorological Center’s Global Environmental Multiscale (GEM) models are included in the study. Errors are compared between coasts and between models. Because synoptically active weather regions (as defined by observed variance in SLP) do not match latitudinally between the coasts, we have selected sample sets comprised of buoy pairs – one from each coast – that have the same observed SLP variance. The effects of recent changes made to these models are examined. The changes include a major update to the GEM in October, 2006, and an operational switchover from Eta to the NAM in June, 2006.

This study focuses on the occurrence of large forecast failures. Documentation includes frequency of errors that meet a “large error” criterion, mean absolute error, mean error (bias), and the distribution of errors using time lines and histograms. Model performance is specifically examined for forecasts associated with high-impact events. In general, the East Coast has fewer large errors and smaller average errors than the West Coast. However, extreme failures do occur on both coasts, some of which are associated with high-impact events, such as the Valentine’s Day Storm of 2007 on the East Coast. The ECMWF model tends to outperform the other models, and there is indication that the GEM model update has resulted in improved forecast ability.