J49.4 AMS 2019 Open Data Distributed on Amazon’s Cloud Service

Wednesday, 15 January 2020: 3:45 PM
157C (Boston Convention and Exhibition Center)
Roope Tervo, Finnish Meteorological Institute, Helsinki, Finland; and M. Sofiev

Finnish Meteorological Institute has implemented Open Data Portal to provide open meteorological time series and grid data. The portal provides weather observations, weather forecast and oceanographic forecast models data. The data is provided in INSPIRE harmonised forms and the portal meets INSPIRE requirements.

The Finnish Meteorological Institute is now building up experience of distributing large amounts of data using cloud services. Since 2017, the Hirlam weather forecasting model covering Europe have been provided in Amazon’s cloud services (AWS S3 Bucket). The objective of the data distribution in the cloud is to increase the utility and effective use of weather and climate data.

In 2019, the Finnish Meteorological Institute is adding new data into the Public Datasets. Global SILAM atmospheric composition and air quality forecast with an unprecedented spatial resolution of 0.2 x 0.2 degree and hourly time step. The forecast is updated daily and spans over 5 days forwards. Predicted quantities include all main trace gases, a variety of primary and secondary aerosols, as well as derived parameters, such as atmospheric optical density. The forecasts are made with the latest emission from vegetation fires derived from near-real-time satellite observations.

The data is licenced with The Creative Commons Attribution 4.0 International license (CC BY 4.0). That is, the users may freely use, distribute and modify the data presuming that the original producer is attributed. The data is available in two different formats. NetCDF is a traditional format used in environmental science. Moreover, the data is converted also into Zarr-format to enable easy and efficient processing in cloud environments with software like Pangeo. While the service is particularly practical for anyone processing the data in the cloud, AWS Public Dataset program provides a great channel to distribute and consume the data for all users.

- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner