Scaling UXarray: Bridging the Gap for High-Performance Unstructured Grid Analysis

Global climate models (GCMs) now operate with unstructured dynamical cores that permit explicitly resolving nonhydrostatic motions and regional resolution refinement. Models that run at individual-storm-resolving resolutions generate large volumes of unstructured grid data that needs to be analyzed at scale. These efforts have introduced the need for scalable analysis algorithms, which have been commonplace for structured grids thanks to packages such as Xarray and Dask. Project Raijin has created UXarray, an open-source Python package that enables the analysis and visualization of such datasets without the need of regridding to structured grids.

The performance of UXarray’s existing implementation of analysis routines has been assessed, which have been primarily written using Numpy. Dask was heavily leveraged to scale UXarray for the analysis of the vast volumes of unstructured GCM output, as demonstrated in the revision of the existing topological aggregation routines and the implementation of a new weighted-averaging routine. Benchmarks on high-resolution unstructured grids at scale were performed to quantify the performance of UXarray on NCAR HPC systems, including Casper and Derecho.

Additional documentation has been written for new routines and package in general, including workflow examples with output from the Department of Energy (DOE) Energy Exascale Earth System Model v2 (E3SM-v2), and how to leverage Dask to achieve optimal performance with UXarray.