THREDDS 5.0: The Good, The Bad, and The Awesome

The THREDDS Data Server (TDS) is a web server that provides metadata and data access for scientific datasets, using OPeNDAP, OGC WMS and WCS, HTTP, and other remote data access protocols. The current stable version of the TDS is v4.6. However, development of the next major version of the TDS, v5.0, is well underway and quickly approaching beta status. In this talk, the major changes to the TDS will be discussed from the point of view of both server and client side changes.

In v5.0, the configuration catalogs and internal state of the TDS has been extensively reworked to be able to scale to large numbers of catalogs, datasets, and internal objects without excessive use of memory. A running TDS can be triggered to reread the configuration catalogs without having to restart, and to reread only changed catalogs, for fast incremental updates. Other features have been added to make writing configuration catalogs more maintainable, including the catalogScan element, and default and standard services. Many of the enhancements are motivated by the use of TDS in the Earth System Grid servers, which can have tens of thousands of catalogs.

The other major enhancement is that the API for Grid datasets is reworked to better support very large data collections, such as those enabled by the new GRIB Collection feature added to TDS in version 4.6. The new API works only with coordinate values (not array indices), which solves various intractable problems that arise when using array index subsetting on large collections.

Future and ongoing efforts in the THREDDS project (netCDF-Java and the TDS) will be discussed including the THREDDS team's support for client-side libraries, specifically Python support.