9A.6
Real time Filtering and Mining of NEXRAD Streams for Mesoscale Forecast and Prediction
Beth Plale, Indiana University, Bloomington, IN; and N. Vijayakumar, R. Ramachandran, X. Li, and T. Baltzer
Weather forecast and prediction models ingest timely regional observational data. But doing so under conditions so time-sensitive (where minutes save lives) that the process has to be completely automated still falls in the domain of active research. That is, middleware systems today that enable data ingest (or assimilation) of time-sensitive regional observational data, in our experience are not sufficiently agile and automated to respond in real time to a developing severe storm. In research conducted under the umbrella of the Linked Environments for Atmospheric Discovery (LEAD) project [2], we are addressing this limitation through a tool and programming interface that enables a scientist to task data mining algorithms to run continuously over one or more NEXRAD streams until a weather phenomenon is detected, whereupon the system will automatically trigger a mesoscale weather prediction model over the region in which the phenomenon is detected.
The tool is designed as a grid or web service, so is pluggable into the workflow system running on the LEAD grid. The programming interface to the tool is the same as is used to access and manipulate data in a database management system – that is, a database query language. Through issuing SQL-like queries, features are extracted from a “database” of continuously arriving NEXRAD data. Sophisticated data mining algorithms are part of the tool suite. The NEXRAD data is obtained by a specialized Unidata LDM stream[1].
In this paper we discuss the mining and filtering tool as it is being used in dynamic mesoscale meteorology forecasting. Whereas in [5] we gave a general model for dynamic query processing, in this paper we provide details of the working prototype of the system that is now undergoing early tests. It couples the Calder system[3], phenomena detection algorithms from the AdAM data mining toolkit[4], and a communication layer that communicates with a custom configured LDM[1]. We report initial performance measurements to show scalability.
Bibliography
[1] Mitchell S. Baltuch, Unidata's Internet Data Distribution System: Two Years of Data Delivery, AMS/IIPS 1997
[2] K. Droegemeier, K. Brewster, M. Xue, D. Weber, D. Gannon, B. Plale, D. Reed, L. Ramakrishnan, J. Alameda, R. Wilhelmson, T. Baltzer, B. Domenico, D. Murray, A. Wlson, R. Clark, S. Yalda, S. Graves, R. Ramachandran, J. Rushing, E. Joseph, "Service-oriented environmens for dyamically interacting with mesoscale weather", Computing in Science and Engineering, IEEE Computer Society Press and American Institute of Physics, Vol. 7, No. 6, pp. 12-29, 2005.
[3] Ying Liu, Nithya N. Vijayakumar, and Beth Plale, Stream Processing in Data-driven Computational Science To appear 7th IEEE/ACM International Conference on Grid Computing (Grid'06), Barcelona, September 2006. (18% acceptance)
[4] John Rushing, Rahul Ramachandran, Udaysankar J. Nair, Sara J. Graves, Ron Welch, Amy Lin, ADaM: A Data Mining Toolkit for Scientists and Engineers, Computers and Geosciences, in press, 2005
[5] Nithya N. Vijayakumar, Beth Plale, Rahul Ramachandran, and Xiang Li, Dynamic Filtering and Mining Triggers in Mesoscale Meteorology Forecasting IEEE International Geoscience and Remote Sensing Symposium (IGARSS'06), Denver, CO, August 2006.
Session 9A, Internet Applications and Cyberinfrastructure
Thursday, 18 January 2007, 1:15 PM-3:00 PM, 216AB
Previous paper Next paper