DART-X: Software Infrastructure for Prototyping in-Memory Data Transfer between Ensemble Data Assimilation and Coupled Earth Systems Models

Data assimilation (DA) is a powerful tool to integrate real-world data into coupled climate models, capturing the complexity of the Earth system and improving prediction accuracy. NCAR’s Data Assimilation Research Testbed (DART) is an ensemble DA tool used for climate predictions with NSF NCAR’s Community Earth System Model (CESM). Traditionally, DART has relied on modifying "restart" files written to disk to influence numerical models, which involves significant I/O and model stop/restart processes, incurring high computational costs on high-dimensional models, even on large supercomputers.

Here we use the National Unified Operational Prediction Capability (NUOPC) layer to enable direct in-memory data transfer between DART and CESM.

I will describe software infrastructure to develop and test the NUOPC cap for DART, enabling full software utilization and integrating the cap into a working system. The challenge is to build a new interface with minimal disruptions to existing features and resolve incompatibilities between different software with extensive codebases. I will discuss decisions made on tools, frameworks, and workflow optimization, including midstream adaptations during the simultaneous development of the cap code. I will also describe practices for resolving conflicts using the build system. Employing this infrastructure facilitates the first prototype for in-memory data transfer between DART and a coupled Earth system model, exploring potential enhanced efficiency and scalability in data assimilation.