Science Data Mangement System Tools for Creating Flexible Systems

Through the support of NASA, NOAA and USGS, we have developed and operated numerous Earth science data management systems. While each of these systems would meet their primary objectives, two primary areas of difficulty would arise. The first was difficulty in extending the system to meet new requirements. The new requirements could result from a new data stream or instrument to support or from a new data delivery mechanism. The second was the growth in maintenance costs during the life of the system including adding new algorithms, porting the software to new hardware or modifying underlying infrastructure software (e.g. new database software). To deal with these challenges, we have developed two technologies that mitigate these problem areas. The first is a science data management system prototype based on a Service Oriented Architecture (SOA). The prototype utilizes a mix of existing system components, including SOA based, and new components to provide the basic functionality for a science data management system. This system decouples system components allowing any to be modified independent of the others. This approach facilitates the quick addition of functionality to support new requirements. It also allows the incremental addition of functionality (i.e. new distribution mechanism) without impacting the existing capabilities. The maintenance costs will be lowered due to this flexibility. The second technology, which will lower maintenance costs, is an algorithm creation framework that reduces the time to create science product generation software by providing libraries to decouple data management functions from the science algorithm code. We have found that 60-80% of the code for a typical science application is data management code and that continuous change of science applications is inherent in research environments we support. Our generic science application framework includes the core data management functionality that is common to many remote sensing applications and was designed to be easily configured to work with many different instruments and science data sets. It has been used successfully in several applications resulting in significant time savings and code reuse. These two technologies will enable science data management systems to be more flexible and operate with lower maintenance costs.