Unified Workflow Tools: A Component Driver and Configuration Toolbox

Unified Workflow Tools: A Component Driver and Configuration Toolbox

Emily Carpenter^1,2, Christina Holt^1,2, Paul Madden^1,2, Naureen Bharwani^1,2, Fredrick Gabelmann^3,4, Brian Weir^5,4

¹Cooperative Institute for Research in Environmental Sciences (CIRES)

²National Oceanic and Atmospheric Administration, Global Systems Laboratory (NOAA GSL)

³Element 84, Inc.

⁴NOAA Earth Prediction Innovation Center (EPIC)

⁵RTX Corp.

Unified Workflow is a versatile Python package that provides command-line tools and a Python API to streamline the configuration and running of common workflow tasks within the Unified Forecast System (UFS).

Historically, each UFS application has created bespoke scripts to drive its workflows, leading to duplication of effort and incompatibility across applications. The UW vision is to unify these disparate configuration layers into a single, cohesive framework that interfaces with existing workflows and workflow managers. The aim is to alleviate common pain points, provide the tools necessary to create a concise and efficient runtime environment, flatten the learning curve to get what you need out of a UFS app, and empower scientists to realize their experiments exactly as they envision them.

In the past year, UW has developed a library of drivers that are each tied to a particular component (e.g. models, pre-processors, post-processors) and handle preparing run directories, staging data files, defining the component’s resource requirements, and executing it, either directly or by submitting a job to a batch system. UW currently has drivers that not only support key UFS Application components, but also planned components, including JEDI and MPAS.

As UW is integrated into UFS Applications, NOAA GSL’s MPAS App has been fully developed using uwtools and demonstrates a simplified model workflow that will support the integration of a standalone MPAS model. On our poster, we will provide an overview of the current library of UW drivers, the workflow toolbox for configuration management, and how users can start interacting with these tools today.