Developing Distributed Data Access Capabilities for the WRF Portal and beyond

Significant efforts have recently been undertaken to understand the state of NOAA's observation systems under a program called the NOAA Observing System Architecture (NOSA). NOSA has focused on identifying NOAA's observing systems as a means to highlight gaps and duplication in current observing systems. However, delivery systems must be developed to locate, access and integrate data into the models, systems and users that require them. These systems will be distributed in nature, in order to account for data located in repositories around the nation and the world. These are also central goals of the Global Earth Observation System of Systems (GEOSS) that is being supported and promoted at the highest levels of NOAA, but development and deployment of these systems are critical. Systems must be developed to integrate metadata information into data catalogues and registries so applications and users who need to locate data will have effective mechanisms to obtain the data they need. In addition, efficient data transport mechanisms must be incorporated to provide timely access to data when required.

The Forecast Systems Laboratory's development of the WRF portal will be used to evaluate distributed web-based technologies in support of NOAA's GEOSS plans. The Weather Research and Forecast (WRF) portal is a java-based web application designed to utilize distributed compute and data resources necessary to conduct systematic test and evaluation of the WRF model. This portal is being developed in collaboration with researchers at the NOAA, NCAR, and the DTC to simplify configuration and running of tens to hundreds of model runs to support model development activities and to perform the evaluation of existing or proposed observing systems using OSSEs and data denial experiments. The ability to run long simulations requires access to high performance computing resources as well as access to high-resolution data streams necessary for model initialization, analysis, visualization and verification.

This presentation will focus on research efforts to evaluate current web technologies that can be used to locate, access and utilize local and remote data sets. Many technologies used to access web data focus on requirements for the interactive user: to display and analyze datasets on a workstation, for example. In this development we also consider application access to data where automated mechanisms are required to obtain data on-demand necessary to run tens to hundreds of model runs. Technologies such as openDAP (formerly DODS), THREDDS, GraDS Data Server (GDS), and web services will be discussed. These capabilities will also be evaluated as a means to understand how NOAA might want to build distributed data systems to serve the needs of its constituents in the future.