HPCW: the High Performance Climate and Weather Benchmark Suite

How well is a High Performance Computing (HPC) system suited for specific complex applications like weather and climate models? The computational performance of supercomputers is usually expressed in terms of their theoretical peak performance or measured with rather simple but easy to handle benchmark codes, like the High-Performance Linpack benchmark, a measure of the floating-point arithmetic computing power. These synthetic metrics enable an easy ranking of the performance of HPC systems. However this ranking has only limited value when it comes to weather and climate science. From another perspective, benchmarks are also used by technology providers in their (co-)design processes, so that the innovation matches benchmarks needs. In that aspect, providing benchmarks that are representative of science is critical for the innovation to best benefit science. To answer this need, we propose the High Performance Climate and Weather (HPCW) benchmark suite, that aims at providing a Domain Specific Benchmark (DSB) focused on weather and climate.

HPCW has been designed and developed under the EU-funded projects ESCAPE and ESiWACE. Moreover, HPCW continues to add new features and reach new platforms thanks to ESiWACE and another new EU-funded project, HANAMI, in collaboration with Japan. This DSB currently consists of widely used European weather and climate models, as well as standalone model components. Such so-called dwarfs are less complex than the full models themselves but represent selected characteristics and associated computational and memory access patterns. HPCW enables reproducible, portable, self-contained and detailed performance comparisons, and integrates easily with third party tools, such as profiling tools. Our ambition is to establish HPCW as a useful benchmarking tool for weather and climate modelers around the world on one hand and supercomputer vendors including technology developers on the other hand by providing a common framework to foster collaboration on co-design, performance optimization and portability aspects.

After introducing the motivation of this initiative, we present the components and the main features of HPCW. We first emphasize on what makes a good benchmark: easily portable, self-contained, reproducible, and shipped with means of verifying that the input and output are correct, of providing performance indicators – including using those of profiling tools – and of extending easily to new models, dwarfs, or versions. Such considerations make HPCW agnostic to the software environment and scheduler of the target HPC system. For instance, the effort of supporting all combinations of compiler versions, vendor libraries (like MPI) and/or models leads us to consider the usage of a package manager like Spack. Then we present some results of HPCW deployments on different HPC systems. We finally conclude with the current roadmap of features, components, and deployments.