21st International Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology

16.3

Networked synchronization of netCDF datasets

Joe Sirott, Sirott and Associates, Seattle, WA; and L. C. Sun and D. W. Denbo

It is now universally expected that scientists in the geophysical data community will have remote access to scientific datasets. Many of these datasets are constantly changing as new information is gathered or generated. The most commonly used protocols for accessing these datasets are ftp, http, and OPeNDAP. All of these protocols assume that the client has uninterrupted, high speed access to the Internet.

In today's world of mobile computing, it can no longer be assumed that clients will have uninterrupted, high speed network access. Further, even if a client has high speed access, network latency in wireless networks can be much greater than in traditional wired connections. One alternative to traditional network based access to datasets is to mirror a remote dataset on a local machine.

This paper describes cdfsynch, an extension to the rsync algorithm that is optimized for efficient mirroring of netCDF datasets over a high latency, low bandwidth network link.

extended abstract  Extended Abstract (16K)

wrf recording  Recorded presentation

Session 16, Internet Applications, Web Portals and Cyberinfrastructure Part II
Thursday, 13 January 2005, 8:30 AM-9:45 AM

Previous paper  Next paper

Browse or search entire meeting

AMS Home Page