The Unidata LDM System: Recent Improvements for Scalability

Since 1995, with the assistance of a cooperative community effort, Unidata has been propagating near-real-time data to participants via its Internet Data Distribution System (IDD). The IDD delivers near real-time meteorological data continuously from multiple sources to educational and government institutions. The distributed software used for this data delivery is the Unidata Local Data Management (LDM) system, a collection of protocols and freely-available programs that provide for reliable event-driven data distribution as well as facilities to select, capture, process, and redistribute data products.

With recent increases in the number of data products available via the IDD, the LDM software would have reached the limits of its ability to keep up with the reliable delivery of data for the IDD without changes in LDM algorithms for data management.

We summarize the current status of the LDM and its use in the IDD, including recent improvements and their effect on the scalability, performance, and reliability of the system. We also discuss remaining LDM limitations and plans to address them to keep data products flowing reliably to the applications and users who need them.