The goal of the MEAD Expedition is the development and adaptation of Grid and TeraGrid-enabled cyberinfrastructure for enabling ensemble or very large domain model simulations coupled with data handling, analysis, data mining, and visualization services. This includes a dynamic workflow and data management environment applicable in many fluid flow modeling situations. The specific applications chosen for MEAD are mesoscale storm and hurricane research and education. MEAD is a cyberinfrastructure proving ground that has been funded for two years by the National Computational Science Alliance, an NSF PACI program. It’s successor, LEAD (Linked Environments for Atmospheric Discovery), will build on MEAD efforts while focusing on real time, on-demand, fault tolerant environments (see LEAD abstract).

Portal Grid and Web infrastructure is beginning to enable launching of hundreds of individual Weather Research and Forecasting (WRF), Regional Ocean Modeling System (ROMS), or coupled WRF/ROMS simulations in either ensemble or parameter mode. Discovery and use metadata coupled to the resulting terabytes of data will then be made available to enable further exploration. Thus, a user of the MEAD workflow will be able to configure and integrate model simulations, manage resulting model and derived data (10’s to 100’s of terabytes), and analyze, mine, and visualize large model data suites in a research (not predictive) context. Finally, very large domain research fault-tolerant simulations will be enabled through grid services and decomposition techniques that can be utilized efficiently on the new TeraGrid architecture.

MEAD efforts have been organized into seven areas and associated working groups. In addition to those funded through the MEAD expedition, there are participants from four other Alliance expeditions, EOT (PACI Education, Outreach, and Training Program), and other unfunded efforts at NCAR, ANL, and NOAA. An overview of activities in each area will be presented. They are

Collaborative Analysis and Visualization (CAV): Existing visualization tools are being adapted for use in the MEAD environment and include stereo GeoWall display, simultaneous display of multiple simulations on a tiled display wall, workflow-driven visualizations, and collaborative analysis and visualization using personal Access Grid technology in collaboration with SWOF. Software includes NCAR graphics, the Java-based Interactive Data Viewer (IDV) built on VisAD, the parallel ParaView visualization system, the Hierarchical Volume Renderer, TeraVision for streaming animation, collaborative Access Grid 2, and a shared image markup tool for visualization and annotation of image data.

Coupling and I/O (CIO): The Weather Research and Forecasting Model (WRF) and the Regional Ocean Modeling System (ROMS) are being coupled and HDF5 I/O is being implemented in both models for use in the Grid environment. Coupling is through HDF5 file transfer and utilizes functions from the Modeling Coupling Tool (MCT). I/O in WRF and ROMS is being handled through a common API (that used in WRF) and will enable writing/reading of files in both netCDF and HDF5. Reading of either type of file through the API will be handled using ESML (Earth System Markup Language). ADAS (ARPS Data Assimilation System) is also being adapted for interpolation to WRF grids and includes assimilation of Doppler radar data.

Data Mining and Machine Learning (DMML): Data mining and machine learning tools are being adapted to help researchers analyze data from 10’s to 100’s of model simulations (ensembles and parameter studies) distributed across the grid. This includes current and new functionality provided through datamining services from D2K and AdaM that include many algorithms and a variety of visualization capabilities. Feature-detection algorithms are being developed and new approaches are being explored such as the use of the grid-enabled MATLAB/Parallel Problems Solver environment and application of a hypergraph clustering algorithm.

Education (ED): Understanding of model uncertainty in weather, including the prediction of hurricane strength, behavior, and human impact, will be aided through curriculum development and provision of simplified MEAD related model, analysis and visualization tools. The materials focus on inquiry-based learning with hands-on experimentation and bring students into the world of the scientific researcher and weather forecaster. This work is being done in collaboration with EOT investigators.

Metadata and Data Management (MDM): Stored data from model simulations carried out in the MEAD environment will be accessible through user-specified and automatically-generated metadata provided through catalogue and Grid Data Services. The user will eventually be able to publish derived data, visualizations, and textual reports/annotations using this system. This work is being done in collaboration with the PACI Data Quest Expedition and in consort with the Earth System Grid effort (NCAR, ANL, and LLNL), the Metadata Catalogue Services (MCS) effort (ISI), and other efforts.

High Performance Grid Computing (HPGC): Improvement of model performance on parallel clusters for WRF and ROMS is being carried out in collaboration with the Performance Expedition using LoopTool for loop fusion, HPCview for compile time performance improvement and Prophesy for run-time performance improvement. The MEAD workflow will include allocation of grid resources using a form of GrADS. A fault-tolerant grid computing paradigm is being implemented using SHMOD for carrying out very large fluid simulations.

Portal Interface and Workflow (PIW): The MEAD-PI interface is being developed for specifying model parameters for suites of simulations that will be executed on the Grid and subsequently analyzed/mined and visualized. Workflow management, including support for coordination of job submission, scheduling of model simulations, analysis, and visualization, and collation of model simulations, is being carried out with software developed in collaboration with the Portal Expedition in a framework referred to as the MEAD Application Grid Services Framework (MEAD AGSF).