Preparations for Next-Generational "Exascale" Data Assimilation at the Met Office

Current data assimilation (DA) capabilities at the Met Office are based on the Observation Preprocessing System (OPS) and VAR DA system, developed and first implemented in the 1990’s. Since that time, the sophistication and efficiency of QC, bias correction, data thinning, forecast sensitivity to observation impact (FSOI) and DA algorithms has undergone continual development leading to many of the major performance improvements in NWP over the past 20+ years. Current DA techniques are based on a six-hourly-cycling hybrid four-dimensional variational (4DVar) algorithm for global NWP and an hourly-cycling (non-hybrid) 4DVar for km-scale UK NWP, with millions of observations being assimilated operationally in each analysis cycle 24 hours a day, seven days a week.

The Met Office has initiated a wide-ranging ‘Exascale’ programme to prepare weather/climate modelling systems for future hardware architectures and an expanding range of Earth System applications (e.g. atmosphere, land, ocean, wave, ice, biogeochemistry, etc.). A key part of this effort, targeted for operational replacement of the current ‘Unified Model’ (UM) system sometime in the mid 2020’s, is the specification of requirements, planning, development and implementation of next-generation observations preprocessing (OP) and DA capabilities suitable for the next 10 – 20 years.

Following a brief overview of current capabilities, this talk will focus on nascent activities to develop next-generation ‘ExaDA’ OP/DA capabilities. An overview of generic requirements (flexibility to algorithm/application, portability to an expanded range of hardware architectures, scalability, efficient interface to model/ensemble, robustness, etc.) will be provided. The significant investment in, and high performance of, current OP/DA capabilities sets a high bar for any candidate replacement system. Some of the key unique features of the Met Office’s current capabilities that provide significant benefits will be described – these algorithms (e.g. 1DVar QC of radiances, simplified linear perturbation forecast model in 4DVAR) will either need to be ported, or suitable alternatives found.

The Met Office is only one of several operational/research centres consider major refactoring/redesign of current OP/DA capabilities in preparation for the Exascale Era. Similarities in requirements for many centres, and the sheer level of effort required, implies the consideration of new collaborations and paradigms for software development. This talk will end with a brief summary of ongoing collaborations (covered further in other talks) with the JCSDA, US Air Force and NCAR to explore the use of the JCSDA’s JEDI framework as a candidate DA framework for Met Office’s ExaDA capabilities.