Efficient multi-GPU implementation of a moving boundary approach in rotor flow simulation using LBM and level-set method

Moving boundary recognition exists widely in the numerical simulation of motion problems in fluid mechanics engineering. Particularly, in rotating machinery flows simulations, a method for handling moving boundaries with high-resolution grids, high computational performance, and efficient implementation on high-performance computing systems is crucial. Based on an in-house lattice Boltzmann method (LBM) solver, this study has developed a moving boundary approach suitable for simulating three-dimensional rotating flows. This method couples a multi-block grid method for local grid refinement and utilizes the level-set method for accurately capturing moving solid boundaries. Moreover, the implementation has been successfully carried out on a desktop- level multi-graphics processing unit (GPU) parallel system. The results show that adjusting the number of GPUs enables flexible scaling of the computational domain size, making this method particularly well-suited for large computational domains in rotating flow problems. Furthermore, the detailed evaluation of parallel GPU performance reveals that the computational performance with nine GPUs in parallel at maximum grid size is 2.33 times greater than that with three GPUs in parallel. Additionally, when the grid size per GPU varies, both kernel functions time and communication time significantly impact performance. The optimized data transfer strategy helps to minimize interpolation overhead and avoid additional communication overhead associated with multi-block grid refinement. The test results show a maximum MLUPS performance of 3074.85 with three V100 GPUs in parallel. Finally, the simulations of flow over three rotor configurations indicate that the proposed implementation accurately identifies rotating motion boundaries and can be applied in real-world rotor flow simulations.