Monday, 10 January 2005
A prototype for the GIFTS information processing system
The Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) instrument is expected to generate data at rates exceeding 1TB/day. The design of a proof-of-concept prototype ground data processing system is presented, which is responsible for turning raw interferograms from the instrument into calibrated, geolocated spectra in near-realtime. This prototype is intended for processing data generated during ground tests, as well as for testing algorithm variations, design alternatives, and implementation strategies. It embodies the core philosophy and architecture ideas from design studies carried out previously for a production-level GIFTS data system. These include a component approach which uses the Unified Modeling Language - Real Time extension (UML-RT) to capture and communicate design decisions, a processing pipeline with pluggable algorithm stages, a flexible architecture implementable on a variety of platforms but targeted mainly for a distributed computing facility such as a cluster, and focus on the careful management and annotation of metadata with a view on complete reproducibility of processing and on fault tracing.