Although incorporating the horizontal diffusion into the atmospheric governing equations is a part of the dynamics of a model, it is based on the computational consideration originally, which is to overcome the nonlinear aliasing or computational instability and smooth the physical fields at the same time. The generally used forms of the 4-order diffusion, either linear or nonlinear, can, however, not properly describe the real physical diffusion processes and thus affect the accuracy of the forecasting and modeling. In this paper, based on the nature of the physical diffusion, the scheme of horizontal diffusion of the Version3 MM5 is improved according to the second law of thermodynamics which controls the irreversible evolutionary direction of a many-body system and the potential ability of the improvement of the new scheme in the simulation accuracy is revealed by introducing the technique suggested in this paper, via a theoretical case and the computational case of the hurricane Andrew that landfell on Florida in 1992.