Computational models based on discrete dynamical equations are a successful way of approaching the problem of predicting or forecasting the future evolution of dynamical systems. For linear and mildly non-linear models, the solutions of the numerical algorithms in which they are based converge to the analytic solutions of the underlying differential equations, for small time-steps and grid-sizes. In this paper, we investigate the time-step sensitivity of three non-linear atmospheric models of different levels of complexity: the Lorenz equations, a quasi-geostrophic (QG) model and a global weather prediction system (NOGAPS). We show that for chaotic systems, numerical convergence cannot be guaranteed forever. The decoupling of solutions for different time-steps follows a logarithmic rule similar for the three models. In regimes that are not fully chaotic, the Lorenz equations are used to show that different time-steps may lead to different model climates and even different regimes. We propose a simple model of truncation error growth in chaotic systems that reproduces the behavior of the QG model error growth for different time-steps. Experiments with NOGAPS suggest that truncation error can be a substantial component of total forecast error of the model. Ensemble simulations with NOGAPS show that the use of different time-steps provides a simple and natural way of introducing an important component of model error in ensemble design.