Tools and Techniques for Developing Atmospheric Python Software: Insight from the Python ARM Radar Toolkit

A large collection of open source software packages and tools exist that can greatly aid in the development of Python software for use in the atmospheric and oceanic sciences. In addition, the scientific Python community promotes the use of a number of software engineering techniques that when utilized claim to produce software which is more robust, readable, easier to extend, and contains fewer bugs. Here we will introduce and discuss the use and benefits of a number of these tools and techniques. In order to show how these tools and techniques can be put into practice, we will provide real world examples of how they were used in the development of the Python ARM Radar Toolkit (Py-ART), an open source Python package for weather radar data.

Py-ART was developed to address the computational needs of the Atmospheric Radiation Measurement (ARM) climate facility after the acquisition of a number of X- and C-band scanning precipitation radars. Py-ART is used to prepare the large amounts of complex data coming from these instruments into data streams consisting of the measured parameters as well as value-added products for dissemination to a large and diverse user base. The package is available freely under a BSD license and can be downloaded from https://github.com/ARM-DOE/pyart/.

The development of Py-ART will be used to introduce and provide an overview of the features and benefits of utilizing the modules in the Scipy stack (NumPy, SciPy, and matplotlib) to provide a base on which scientific Python software can be developed, without the need to write low-level numerical routines. Other Python packages that may be of interest to the atmospheric and oceanic fields will also be discussed. Using tools such as F2PY or Cython, it is possible to interface legacy C, C++ or Fortran code with Python. These tools can also be used to write faster, optimized routines that can be called from within Python. Examples from the Py-ART code base of the performance gains possible when using these tools will be presented. In addition, we will discuss the use and benefits of some modern software engineering practices as applied to the development of scientific Python software including version control, testing, consistent coding styles and the use of embedded documentation. These “best-practices” for software development have been widely embraced by the core scientific Python packages and software that exists to make following these practices as painless as possible will be discussed.