Modeling 3-D Radiative Fluxes within the PALM-4U Microscale Urban Climate Model

Modeling radiative fluxes explicitly in microscale atmospheric simulations, especially in complex urban environments, presents new algorithmic challenges, which are not encountered in mesoscale modeling of radiative fluxes. The presence of mutually visible surfaces separated by large horizontal distances means that there are direct radiative interactions among arbitrary parts of the modeling domain, which deviates from a traditional parallelization scheme, where the modeling domain is divided horizontally into subdomains and most of the interaction and consequently data exchanges happen between adjacent subdomains. This problem is further emphasized when plant canopy is modeled as fully three-dimensional semi-transparent objects.

The Radiative Transfer Model (RTM) in the PALM-4U microscale urban modeling system represents individual surface elements (faces), for which the radiative fluxes and other processes are modelled, using linearized arrays within the memory of the process that models the respective subdomain. The radiative fluxes between mutually visible face pairs are determined using view factors, which represent the fraction of view from each face occupied by other individual faces. The remaining fraction of view, which is necessary for modeling directional and diffuse solar irradiance together with longwave irradiance from the sky and longwave radiation emitted towards the sky, is called the sky view factor. The RTM models plant canopy explicitly within the 3-D grid: the grid cells occupied by plant canopy cause partial absorption of the passing shortwave and longwave radiation depending on their leaf area density (LAD), and they also emit longwave radiation depending on their temperature. These processes are quantified using so-called canopy view factors.

The mutual visibility among faces and the values of the view factors, sky view factors and canopy view factors are determined using a specially designed raytracing algorithm. This process is not only computationally expensive, it also requires a lot of interprocess communication in a highly parallelized setup. Because the geometry of terrain, buildings and other obstacles is modelled as constant in time, it is possible to perform the raytracing in advance during model initialization. The application of the precomputed view factors on actual radiative fluxes within time-stepping is then straightforward with much lower demands for CPU time and interprocess data transfer.

RTM version 3.0 offers two schemes for discretization of each face’s view. The legacy per-face discretization scheme performs raytracing and establishes view factors individually for each pair of mutually visible faces with optional limitation by distance and viewing angle. This scheme works well for small to medium setups, but it does not scale well for large domains and/or high resolution simulations with horizontal grid dimensions in the order of thousands of grid points. The current angular discretization scheme, available since RTM version 2.5, discretizes view from each face using a constant set of azimuth and elevation angles, limiting the spatial resolution of mutual radiative exchange only among the surfaces that are very distant or under shallow angles, i.e. which do not represent a large portion of each other’s view. This scheme scales well even in the largest setups allowed by the atmospheric model, without any noticeable drop in accuracy. (In fact, accuracy actually improves for near faces due to better representation of angular sizes.) Since the maximum amount of view factors per face is constant under this scheme, the total amount of surface view factors grows with O(n²) and the total number of canopy view factors grows with O(n³) if the size of the grid is increased by a factor of n in each dimension, therefore it scales in pair with other processes in the PALM-4U model. The angular scheme also uses a newly optimized raytracing algorithm, which traces all the rays directed in one azimuth concurrently, bringing much better granularity of interprocess data exchange, improved memory access patterns and more efficient computation overall. Both raytracing algorithms in RTM utilize MPI one-sided communication with passive target synchronization. This advanced MPI feature allows a process that traces rays passing through some other process’ subdomain to query data (e.g. 3-D arrays with LAD of plant canopy) from that process’ memory without its synchronized cooperation.

Optimizing the algorithms for different domain sizes and different amounts of parallel processes leads to different implementations of certain sub-tasks. In currently used scenarios, it is often not possible to fit any domain-sized 3-D structure in the memory of each parallel process, however this is still possible with the domain-sized 2-D terrain elevation array and with the global linearized arrays of per-face values. This property is utilized both in the raytracing algorithms to significantly decrease the amount of interprocess data exchange, and also in the time-stepping code where a potentially more optimized data exchange pattern may be utilized at the expense of higher memory consumption. An extension is currently being prepared for the extremely large cases where even the domain-sized 2-D arrays could not be stored in each parallel process.