Using Fermat Number Transform to Accelerate Convolutional Neural Network

Convolutional neural network (CNN), has achieved a significant breakthrough on image recognition and natural language processing. However, CNNs still suffer from the prohibitive computation complexity. Many efforts have been made to reduce the arithmetic complexity of direct convolution Aiming at lower multiplication complexity, an efficient convolution architecture based on Fermat number transform (FNT) is proposed in this paper. We first present the FNT algorithm and Overlap-and-Add (OaA) method. To cope with convolution computation in CNNs, the FNT is extended to two-dimensional (2D) FNT and corresponding calculation methodology is introduced. The pipelined FNT architecture is also proposed for efficient realization of FNT. Then an overall architecture of CNN accelerator is given based on OaA FNT. Complexity analysis has demonstrated that the proposed OaA FNT convolution method reduces 5.59x and 2.56x of multiplication complexity compared to direct convolution and OaA FFT method. We also conduct evaluation on the FPGA platform Xilinx Virtex-7 XC7VX485t and comparison is made with respect to convolution throughput and resource efficiency (GOP/s/DSP). The proposed architecture achieves 1.41x convolution throughput with 3.05x less DSPs compared to the state-of-the-art design. 4.29x improvement is gained in terms of the resource efficiency.