Multi-Objective Concurrent Kernel Scheduling for Multi-GPU Systems

GPUs are now being widely used in many applications to speed up the computations. Due to its unique architecture, GPUs are well suited to run embarrassingly parallel applications such as deep learning and big data analysis. However, GPUs are power hungry devices, and many applications are not able to fully utilize the resources of a GPU, resulting in energy wastage in multi-GPU systems. To tackle this challenge, recent advancement has made it possible to run multiple kernels concurrently on a single GPU to share its resources. This however comes with a performance penalty due to the interference of concurrent kernels while competing over shared resources. In the presence of such impacts, mapping and scheduling of multiple kernels on multiple GPUs in order to fully utilize the GPUs, minimize the energy consumption, and reduce interference and performance degradation is challenging. In this paper, we propose a multi-objective kernel scheduling approach for multi-GPU systems, which considers both energy consumption and performance and supports concurrent kernel execution. To solve the scheduling problem, due to its large solution space, a GA-based scheduler called CK-GA has been used that can achieve desirable solutions at a reasonable pace. We also propose a heuristic scheduling algorithm called CK-HEFT based on the well-known HEFT algorithm for concurrent kernel execution on shared GPUs. Experimental results on several multi-GPU architectures show that the proposed CK-GA scheduler has obtained more than 14% improvement in performance, 30% improvement in quality of service and more than 18% improvement in energy consumption on average compared to similar approaches.