Performance Analysis and Optimization of the Weather Research and Forecasting Model (WRF) Advection Schemes

The Weather Research and Forecasting (WRF) model is an open source mesoscale numerical weather prediction (NWP) system extensively used in atmospheric research, operational forecasting, and educational settings with more than 36,000 users from over 150 countries. Since WRF is a computationally expensive model with a wide variety of application and such a vast user base, any performance optimization of the model is of high priority for the WRF developers. This study is focused on several optimizations which will reduce the computational costs associated with WRF and WRF-Chem simulations.

Allinea MAP, TAU (Tuning and Analysis Utilities) and Intel Vtune Amplifier XE were used for profiling and detecting the hotspots of the WRF codes. The profiling results show that advection is the most time-consuming routine in the ARW (Advanced Research WRF) dynamic solver for many WRF and WRF-Chem simulations. For further investigating the hotspots, the advection code is extracted from the WRF Fortran code as a separate kernel. WRF model uses the Runge-Kutta (RK) 3rd order for integrating the time, and 2nd through 6th order advection schemes in both horizontal and vertical directions. This basic RK scheme will produce negative values and oscillations near sharp gradients due to the numerical dispersion errors. Positive definite and monotonic flux limiter schemes are used for reducing the negative values and spurious oscillations from the dispersion errors.
We designed a testing framework for optimizing, compiling, profiling, and evaluating the kernel with different Fortran compilers including Intel, GNU, and PGI compilers. Several optimization techniques including exposing more vectorization and improving the cache hit rates are utilized on the advection modules, and the outputs are verified. Reasonable speed-up can be achieved on different compilers using the modified advection kernel. These source code modifications were tested for bit-for-bit results and integrated back to the WRF source code. These optimizations will widely benefit the large community of research and operational forecasters by reducing the computation costs.