Accelerating Research to Operations with the Joint Effort for Data Assimilation Integration (JEDI) and Skylab Systems

The Joint Effort for Data assimilation Integration (JEDI) is an innovative data assimilation (DA) system for Earth system prediction. JEDI is being co-developed by the Joint Center for Satellite Data Assimilation (JCSDA) and its partner agencies NOAA, NASA, the U.S. Navy and Air Force, and the UK Met Office. A defining characteristic of JEDI is the broad use of shared algorithms (including, but not limited to, observational data filters, quality checks and operators; model data interpolation and DA cost minimization functions) brought about by the highly collaborative development effort of JCSDA and its partner agencies. JEDI also provides a framework that enables the addition of a new forecast model or usage of observational data from a new instrument in a relatively quick and easy manner.

Skylab is a generic testbed for JEDI that enables the user to rapidly conduct DA-oriented science experiments. Skylab provides a workflow system that automatically configures (from a clear and concise input specification) and executes the logical steps for running a cycling DA flow. Skylab currently handles forecast and assimilation of atmospheric, oceanic, soil moisture, sea ice, snow, aerosol and trace gas quantities with more to come in the future.

This presentation will discuss how the features of JEDI and Skylab work to accelerate the transfer of knowledge and techniques from research to operations. In addition to the architecture of JEDI and Skylab, a solid foundational infrastructure is another key aspect of these systems that enables rapid development and integration of new features which will also be discussed. JEDI and Skylab infrastructure includes systems for software stack management, workflow management and the handling of massive amounts of model and observational data; along with agile development methodology, continuous integration and delivery (CI/CD) and cloud computing and data services.