Continental Scale Data Assimilation Using Python

An efficient two-dimensional variational analysis system developed using Matlab was rewritten in Python. Over 20,000 surface observations of 2-m temperature, 2-m dewpoint, and 10-m wind observations are used to create hourly analyses of wind, moisture, and temperature on a 2.5 km grid over the continental United States (1377 x 2145 grid points). Key advantages of developing the analysis code in Python included: open source; object oriented; and the availability of libraries that simplified code development (e.g., numpy, scipy, and matplotlib).

The analysis system requires minimizing a variational cost function in observation space using the generalized minimum residual method. The analysis system requires ~ 8 gbytes of memory and is run on 8 dual core processors. The combined analyses of temperature, moisture, and wind are completed in ~26 minutes, slightly longer than required using Matlab. However, rather than relying on sparse matrices in Matlab, only the required elements of large arrays are stored in dictionaries as a function of the observation identifier. Additional benefits include efficiently storing the analysis grids in compressed hdf5 format using pytables.