In ensemble forecasting with simple models, one usually chooses a number of initial conditions as centers of ensembles, and monitors the evolution of a large number of members within each ensemble to evaluate forecast errors. When the forecast models become increasingly complicated, however, one would only be able to afford ensembles with small number of members, sacrificing estimation accuracy of forecast errors. To improve accuracy due to small size ensembles, we propose a pseudo-ensemble approach, i.e., to exploit the ergodic property of the climatological solution to get information equivalent to what can be obtained by actually constructing ensembles with huge number of members. Specifically, we explore the possibility of constructing pseudo-ensembles from the climatological solution to get all the information needed for quantifying information loss in ensemble forecasting. The feasibility of the proposed approach is demonstrated by general theoretical analysis and numerical simulations based on simple models. To achieve tremendous savings in computational time, data storage, and labor, it is of fundamental importance to systematically sort out which aspects of ensemble forecasting can be sufficiently well characterized by the proposed pseudo-ensemble approach.