The Progress and Trends of FPGA-Based Accelerators in Deep Learning

With the coming of big data era, the technology of deep learning plays a critical role in extracting the meaningful information from the massive data. Also, it has been widely applied in some domains, such as computer vision, speech recognition and natural language processing. As far as we all know, deep learning algorithms have a large number of parameters and relevant matrix multiplication or multiply-and-add operations with the simple computing model. At the same time, researches and industries need higher and higher accuracy, the complex models which have more and more weights won the image classification and object detection contest. To speed up the inference and training of deep learning becomes much more important. This paper mainly reviews one of the approaches, accelerating deep learning on FPGAs. Firstly, this paper introduces the deep learning algorithms and relative characteristics especially the convolutional neural network and recurrent neural network and why the FPGA approach can fit this problem, what people can be benefit compared with CPU only or GPU and what people should do to enable it. After that, this paper analyzes the challenges of accelerating deep learning on FPGAs. Additionally, the CPU-FPGA is the most welcome architecture at present, but there're different methods to set up a usable platform. And for CPU-FPGA platforms,data communication is one of the important factors for affecting the performance of acceleration. Based on these, this paper introduces different methods from the aspects of SoC FPGA and standard FPGA, and comparatively analyzes the differences of the data communication between CPU and FPGA of the two methods. For different engineers and developers, the development environments and tools of accelerating deep learning on FPGAs are presented in this paper from the aspects of high-level language,such as C and OpenCL, and hardware description language, such as Verilog. It is not hard to use techniques which require many complex low-level hardware control operations for FPGA implementations to improve performance by using hardware description. But that always requires a working knowledge of digital design and circuits. In contrary to hardware description language, using high-level language allows engineers and developers to have more freedom to explore algorithm instead of designing hardware architectures and it is suitable for researchers who need quickly iterate through design cycles and software developers who have no knowledge of digital design. And on the basis of it, the FPGA-based accelerator for deep learning algorithms is reviewed from the hardware architecture, design methods and optimization strategy in detail. Three typic hardware architectures are presented for convolutional neural network, and hardware architectures of two typic models for recurrent neural network are introduced. Design methods are reviewed from the goal of accelerator, the way to build model of algorithms and how to find the best solution. As for optimization strategy, it is presented from the processer and memory communication, which are critical factors for improving performance of Accelerator. Finally, this paper prospects the research on FPGA-accelerated deep learning algorithms from the aspects of the development environments, process communications between CPU and FPGA, better compression model methods and cloud application with FPGA. © 2019, Science Press. All right reserved.