CCMC: Heliophysics Modeling Collaboration and Productivity with Cloud Computing

The Coordinated Community Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) serves as a vital hub for the heliophysics community, providing a collaborative platform for space weather model research and forecasting. The CCMC engages with the heliophysics community for development, testing, and public use of over 80 space weather models, bridging the gap between research and operations. Our resources are available both on-demand and continuously updated with real-time data, and cover sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions.

For many years, CCMC has used cloud computing to enable collaboration with scientists, modelers, and researcher software engineers world-wide. This shift to cloud high-performance computing (HPC) has enabled more flexible, scalable, and accessible resources for space weather modeling and data dissemination.

To support its cloud-based HPC infrastructure, CCMC has embraced the "DevOps mindset" to create and maintain the computing infrastructure and software stack (compilers, libraries, and complex dependencies). We maintain a hybrid infrastructure with both on-premises datacenter and multiple services from Amazon Web Services (AWS). Our team uses a variety of open source tooling to operate our HPC environment for on-demand model runs and real-time analysis from scientific instrument observations. The tools discussed will include AWS ParallelCluster, JupyterHub, Airflow, Spack, Grafana, Prometheus, Terraform, Docker, and Apptainer/Singularity containers. This presentation will explore the benefits and challenges from our implementation with a focus on administrative productivity (removing complexity), data volume and variety challenges, and computing run-time performance.