Extremely High-Resolution Weather Model Simulation, Data Processing, and Visualization

Achieving highly accurate and detailed weather forecasts depends heavily on high-resolution simulation and visualization of numerical weather models. High resolution, however, is challenging to achieve, even on the world's most powerful supercomputer systems. We perform a highly localized severe weather simulation using the Weather Research and Forecasting (WRF) model with unprecedented high spatial and temporal resolution. Specifically, optimized parallel WRF one-way nested runs are carried out with about one billion mesh points around Chicago O'Hare International Airport at the resolution of 167 m in the horizontal direction, 90 m on average in the vertical direction, and 3 s in time. Animated results in terms of water mixing ratios, wind components, radar reflectivity, etc., are created and analyzed. The entire work is implemented with thousands of processors on TACC's 10 PFLOPS Dell Linux Cluster Stampede.

Advanced memory management techniques, deliberate I/O workflow, and sophisticated data processing and visualization techniques are well designed for this research based on existing infrastructures and available resources. During the simulation, we collected system-wide performance data, especially the memory usage information during our WRF runs to understand the memory usage pattern of the model. We then reconfigured the SLURM scheduler on the Stampede supercomputer and carefully managed the memory allocation to satisfy all memory requirements through the work. In addition, we took advantage of the local disk space and Stampede's Lustre parallel file system, then created an optimized WRF I/O workflow. Our optimized parallel I/O workflow dramatically raised the I/O performance and cut the load associated with metadata requests of the shared file system. Furthermore, we also scheduled thousands of sequential jobs to regroup and merge intermediate results and created target data files for visualization. The target data files were then converted to the VTK format and loaded into Paraview for visualization.

The resolution of our simulation in time and space is beyond almost all similar weather simulations as we are aware of. Benefiting from the crucial resolution, meteorologists are able to observe subtle weather changes at local areas. Furthermore, all these techniques are applicable to a great deal of memory-intensive and/or I/O-intensive applications, high-resolution simulations, and supercomputer platforms.

Our simulation results are being compared with other observational and computational results by our Raytheon and NCAR collaborators for further validation. Some follow-on studies will be performed over other domains of our interests. We will also investigate WRF's performance benefits from Intel Xeon Phi coprocessors as well as advanced parallel I/O applications in the near future.