Integration of LEAD and WRF Portal technologies to enable advanced research, operations and education in mesoscale meteorology

Imagine configuring an advanced, non-hydrostatic numerical weather prediction model (WRF-ARW) through an elegant graphical user interface, laying out multiply-nested grid domains, specifying physics and numerical techniques options, selecting analyses for initial/boundary conditions, choosing which sources of real-time meteorological data (including NEXRAD Doppler radar data) to assimilate, spawning an ensemble of simulations on a vast array of remote supercomputing clusters, then creating richly textured 4D visualizations of the output as the model runs – all through a web browser from anywhere in the world. This is not a fanciful daydream; it is now possible to do all of this, plus unleash event-triggered dynamically adaptive forecasts, through the Linked Environments for Atmospheric Discovery web portal (http://portal.leadproject.org).

LEAD is a National Science Foundation Large Information Technology Research project that empowers researchers and students with capabilities heretofore available at only a few major universities and research or operational centers, and it does so using a service-oriented architecture similar in many respects to the familiar Amazon.com, where storage and compute resources are provided via “the cloud.” By managing the complexity of interoperative cyber tools and providing flexibility and ease in how they can be linked, LEAD allows users to focus their time on solving the science and engineering problems at hand, providing a means for more deeply understanding the tools and techniques being applied, rather than struggling with the nuances of data formats, communication protocols, software installation, and job execution environments. LEAD has completed five years of research, including the development and testing of a prototype system, as well as operational testing in collaboration with the NOAA National Severe Storms Laboratory and Storm Prediction Center.

The project has propelled research in cyberinfrastructure, virtual organizations, and large-scale software frameworks that support analysis pipelines (i.e., user-assembled workflows) that contain a mix of community and custom models and execute on selected TeraGrid resources under a community account. We describe in this paper several recent developments including a collaborative effort with NOAA/ESRL to merge the best aspects of the LEAD portal (ability to submit large simulations across the TeraGrid, multi-level fault tolerance and recovery, drag-and-drop workflow assembly, data mining and feature extraction) with those of the WRF portal (desktop Java GUI with horizontal and vertical domain editors, support for multiple/two-way interactive/moving vortex-tracking WRF nesting, diff tool for comparing workflows), while creating new portal capabilities (namelist file error-checking/validation tool to prevent input and configuration errors, vertical grid-stretching tool, ability to edit compile and manage application source code). LEAD is now on the precipice of becoming a persistent cyberinfrastructure for mesoscale meteorological research and forecasting. It will soon deployed as a formal community facility funded by multiple organizations and made openly available to support research, education, and operational experimentation, including the emerging concept of “warn on forecast” vs. “warn on detection”.