Interactive Visualization of Global Long-Term Climate Data in a Web Application Using Python and Zarr in Amazon Web Services (AWS)

An ever-present challenge to developing climate-data-driven web applications involves deciding how to translate terabytes (TB) of information into insightful visualizations. Some products pre-aggregate their dataset and limit the temporal extent by which a user can interrogate it. Others limit the scope of what can be viewed at a given time, generating regional static images on customer request. Many come with an inherently limited focus, assessing a single hazard (flooding, fires, heat stress) through proprietary indices, obfuscating the underlying raw values for parameters like temperature, precipitation and wind. Maxar’s ClimateDesk platform allows users to access more than 10 TB of data including up-to-date climate models from the major modeling centers. These data facilitate map-based visualizations of daily data from present to 2100 including on-the-fly temporal aggregation and point-based timeline visualizations of a dozen meteorological parameters for any point on the globe.

To enable this flexibility, we assessed the feasibility of extracting custom spatiotemporal subsets of data from 1 km and 25 km daily data. Our resulting pipeline leverages the cloud-optimized query speed of the Zarr file format, the extensive open-source GIS tools available in Python and NodeJS. The application’s backend and frontend capabilities are delivered using the scalability of AWS with custom Docker container instances. The software allows us to achieve the delivery of data in JSON format as well as expose an OGC-compliant endpoint that builds from cloud-optimized GeoTIFF imagery. This presentation provides an overview of the techniques used in efficiently transforming user requests into slippy map tiles and time series charts, rendering data from a daily-updating suite of subseasonal, seasonal, decadal and centennial scale climate models.