In this work we examine the role that the observed intraseasonal variability can play in controlling and maintaining ENSO variability. To this end, we force an intermediate coupled model of El Nino-Southern Oscillation (ENSO) with observed stochastic forcing which is defined as the part of the atmospheric variability that is apparently uncoupled from the ocean. The stochastic forcing is estimated from 50 years (1950-2000) of NCEP/NCAR reanalysis of surface winds and net heat flux, 31 years (1950-1981) of reconstructed Sea Surface Temperatures (SST) and 19 years (1982-2000) of Reynolds SST in the Tropical Pacific. The part of the surface atmospheric variability that can be linearly related to variations of SST is estimated using the singular value decomposition of the covariance between the atmospheric fields (wind or heat flux) and the SST, and is then substracted from the atmospheric data to recover the stochastic component of the ocean surface forcing. Principal component analysis of the stochastic component shows no preferred mode of variability, exhibits decorrelation times of a few days, and has a spectrum that is indistinguishable from red noise. A 50-year stochastically-forced model integration shows some similarities with the observed equatorial SST. The robustness of this result is checked by performing different sensitivity experiments. Using the ideas of generalized stability theory, the dynamically important contributions of the stochastic forcing are isolated from the rest and it is shown that most of the variability in the stochastically forced run is produced by stochastically-induced Kelvin waves in the western Pacific. The free parameters of the model are chosen so that the coupled system is asymptotically stable (with a decay time of about 3 years). Therefore, these results support the hypothesis that a significant fraction of ENSO variability may be due to stochastic forcing.