GPU-ACCELERATED IMPLICIT TURBOMACHINERY FLOW SOLVER USING MULTIPLE MPI COMMUNICATORS

This paper describes a procedure to port an unstructured CFD solver to GPU for turbomachinery simulations via NVIDIA OpenACC. Matrix-free implicit time stepping is used for robust and efficient time integration. Data movement between the device and the host is managed explicitly and is minimized for optimal performance. For multiple GPU computations, a flexible multistage decomposition is devised. The computational domain is first divided into different components (or sub-domains), each of which represents an physical entity, such as a bladerow. For each subdomain, a graph partition is then performed. Data exchange among multiple GPUs are devised to accommodate such domain decomposition strategy. For halo exchange within each sub-domain, data can be sent directly from one GPU to the other one if permitted by the hardware. To exchange data among different sub-domains, a general purpose coupler is devised to exchange data among the GPUs via the host. In terms of computational speed, for a socket-to-socket comparison (one V100 and 20 CPU cores), the GPU code shows a global speedup of 8.1 for double precision and 10.1 for single precision. Furthermore, the GPU code is 5.5 and 6.9 times more energy efficient for double and single precisions, respectively. The GPU code is validated on Rotor 37 and a high speed multistage compressor. Good agreement with rig data is observed for both cases. The multistage compressor case has roughly 17 million cells and it takes the GPU code 8 hours to produces 4 revolutions using 4 V100. The developed GPU code represents a flexible computational framework for future multi-fidelity/physics simulations on heterogeneous computing environment.