Successful Strategies for Overcoming the Obstacles in Acquisition, Management, and Analysis of CMIP5 Data

The complexity, volume, and distributed nature of CMIP5 data make it one of the most challenging data collections to acquire and manage. In spite of this, CMIP5 data are in great demand, and many users are struggling with the difficulties in getting the data they need. The author will present strategies that are being used to overcome the challenges CMIP5 data users face: authentication, searching for published data that match a list of desired experiments and variables, acquisition of wget scripts, managing wget script execution and the high wget failure rate, retention of critical metadata not present in the data files, version control, local data management, and setting up the data for analysis and visualization using GrADS. All these strategies exist in an automated workflow that is completely independent of any browser interface.