Parallel Singular Value Decomposition on Heterogeneous Multi-core and Multi-GPU Platforms

Singular value decomposition (SVD) is one of the most fundamental matrix calculations in numerical linear algebra. Traditional solution is the QR-iteration-based SVD algorithm on CPU, and it is time-consuming. Nowadays, Graphics Processing Units (GPUs) are suited for many general purpose tasks and have emerged as low price and high performance accelerators. In this paper, the parallel-friendly divide-and-conquer approach is employed to accelerate SVD algorithm on the heterogeneous multicore and multi-GPU systems. Two mechanisms are designed to make good use of the computational resource on the heterogeneous system, including two-layer divide-and-conquer and coordination between CPU and GPU. The experimental results show that our algorithm is faster than Intel MKL with four CPU cores, and reaches 45 times speedup with four NVIDIA GTX460 GPUs over LAPACK. Our implementation can also achieve about 1.5 times speedup by doubling the number of GPU devices.