Ideally in NWP, all ensemble forecasts would be Gaussian, and hence it would be possible to completely describe the full forecast distribution with only its first two moments. Real ensemble forecast distributions can be far from Gaussian though, often bimodal or even multimodal. Probabilistic forecasting is ultimately the correct strategy given these distributions, but for the operational forecaster who must issue a deterministic forecast, the best approach to take when faced with a bimodal ensemble is not at all obvious. However, the ensemble forecast distribution should be able to provide useful information that can help one choose a forecast strategy, or at least give some information about the likely distribution of forecast errors. This premise is explored here in the context of a simple chaotic system that can exist in one of two stable equilibria, and shows long period variation by shuttling between the two. The predictability of these “regime transitions” is evaluated using ensemble forecasts, with special emphasis on utilizing characterizations of the forecast distribution to provide a priori information on the expected error distributions for different types of deterministic forecasts. A probabilistic forecasting approach using these bimodal ensembles is also evaluated in order to gain some insight into the relative merits of deterministic and probabilistic forecasts in a very difficult forecast situation.