A Python Roadmap for Accessing and Interfacing with Big Climate Data in Cloud Storage Providers

The democratization, growth, and wide-spread adoption of the Python programming language and its open-source libraries are empowering organizations, businesses, professionals, researchers, educators, and students alike to work directly with big climate data repositories in the cloud. This open-source movement, coupled with Open Data initiatives sponsored by large technology providers like Amazon, Microsoft, and Google is ushering in an age of data democratization from government agencies and organizations in the cloud. This paper provides a fundamental roadmap to help beginners of all levels, across all industries, and possessing any professional background to connect with, access, process, visualize, and conduct analysis on big climate repositories stored in the Amazon S3, Microsoft Azure, and Google Cloud Storage providers. This paper discusses the motivations and applications driving the adoption of cloud object storage services for climate datasets, including common file types, data formats, data archives, and examples of publicly available climate datasets stored in these cloud providers. Examples of these big datasets include remote-sensing and monitoring station repositories like CMIP6, GHCN, RTMA, CMORPH, and more. Several Python libraries and big data processing strategies are investigated to assist with accessing, visualizing, and efficiently handling these big climate datasets stored in cloud storage providers.