Satellite Data Access and Processing in a Federation of Clouds

EUMETSAT defined in 2016 new data services strategies recognising the changing environment and a paradigm shift of the users behaviour and expectations with a corresponding evolution of user needs. Serious considerations were taken regarding the availability of “big data” technology and policy aspects related to new services and making use of these technologies. The implementation of this new strategy was accomplished via establishing a roadmap for future EUMETSAT data services resulting in the implementation of an entire new service portfolio. This portfolio now offers services that offer an entire service stack, enabling the user to work on a single online data lake based on a EUMETSAT developed harmonised data access service, on top of which self-provisioned processing is possible, whilst benefiting from other portfolio services such as online data access, data customisation tools, data visualisation services using single sign on and authentication in an integrated manner. This new service portfolio implements the “users to the data” and “processing at data location” principles. The services are hosted on a cutting edge multi-hemisphere private on premise cloud with external elasticity and an elaborated Hybrid Cloud Management System across diverse cloud stacks. In order to allow best collaboration with other data centres and data holdings EUMETSAT developed its federation concept, which builds on the achievements of the harmonised data access service together with the HCMS layer. This federation provides an “unseen before” service to end users by enabling the user to access data and services from federated data centres in a transparent and harmonised way – essentially expanding from “user to the data” towards “user to multiple data”. This paper will explain the EUMETSAT new service portfolio as well as the federation concepts and the associated benefits in more detail.