Expanding Access to Advanced Scientific Workflows: The NSF Unidata Science Gateway Leverages Innovative Capabilities of the NSF Jetstream2 Cloud for Atmospheric Science Education

NSF Unidata is a diverse community of education and research institutions with the common goal of sharing Earth Systems Science data and the tools to access and visualize that data. For more than 40 years, the NSF Unidata Program Center has been providing data, software tools, and support to enhance Earth Systems education and research. Funded primarily by the U.S. National Science Foundation (NSF), NSF Unidata is one of the University Corporation for Atmospheric Research (UCAR)'s Community Programs (UCP). The NSF Unidata Science Gateway is an online space where the academic and research communities can collaborate, share resources, and learn from each other in a classroom, workshop, or virtual setting. The Gateway combines NSF Unidata technologies with the NSF Jetstream2 cloud computing platform and open-source software to create an environment where community members can access, analyze, and visualize real-time and case study Earth Systems Science data easily and efficiently.

Since 2020, the NSF Unidata Science Gateway has hosted pre-configured PyAOS (Python for Atmosphere and Ocean Science) JupyterHub servers to facilitate the learning experiences of over 1,830 students across 24 universities, regional workshops, and AMS workshops. These servers provide ready-to-use, cloud-based environments that eliminate the need for students to install complex software on their personal devices. By leveraging the NSF Jetstream2 cloud, these JupyterHub instances ensure consistent and efficient access to advanced tools and datasets regardless of the user’s hardware. This approach not only streamlines the education process but also democratizes access to state-of-the-art scientific workflows, enabling students and educators to focus on learning and analysis rather than technical setup. The flexibility of these cloud-based JupyterHub servers also supports a variety of instructional formats, from traditional classrooms to remote learning to workshops and short courses, making it an invaluable resource for atmospheric science education.

Building on our ongoing efforts to expand the NSF Unidata Science Gateway’s product offerings, the 2021 arrival of the NSF Jetstream2 cloud has unlocked the potential for significantly enhancing the computational capabilities we offer to our community, especially in the areas of GPU and Dask computing. Jetstream2 introduced a new class of virtual machines equipped with various types of GPUs. This development has opened up new possibilities for high-performance computing, notably in fields such as artificial intelligence and machine learning (AI/ML). Through the NSF Unidata Science Gateway, we now offer this advanced GPU computing resource to our community via customized PyAOS JupyterHub servers, which leverage high-level Python libraries such as TensorFlow and PyTorch to facilitate GPU-accelerated workflows. In addition to GPU computing, our collaboration with Jetstream2 staff has enabled us to launch Dask-enabled JupyterHub servers. Dask is a Python library for parallel computing, allowing for the efficient analysis of increasingly large datasets. This capability is particularly valuable for the atmospheric science community, where handling and processing these extensive datasets is an ever-growing challenge.

This presentation will highlight two case studies that demonstrate the practical applications of our advanced computational tools. The first case study involves our support of a Master’s thesis project focused on the predictive capabilities of AI/ML in atmospheric science, specifically using convolutional neural networks (CNNs) to simulate and predict storm patterns. The second case study showcases our collaboration with an atmospheric science professor, where Dask and Xarray were employed in her class to efficiently analyze large atmospheric datasets, illustrating the power of parallel computing in this scientific discipline.