An Adaptive Row-based Weight Reuse Scheme for FPGA Implementation of Convolutional Neural Networks

There is an increasing need to implement the Convolutional Neural network (CNN) with an FPGA thanks to its design flexibility over an ASIC and low power consumption over a GPU. The size of the network and the resource of the target FPGA board should be considered to deploy the CNN Network successfully. However, previous works use the fixed dataflow which is not optimized for each layer. As a result, high on-chip buffer utilization and frequent memory access are required. The row-based weight reuse scheme is efficient in reducing input/output buffer size. However, it causes resource underutilization for layers with small feature maps size. This paper proposes an adaptive row reuse scheme by applying each level of row-reuse for each layer depending on its characteristic. Finally, the proposed design is implemented with a Xilinx KCU1500 board, and the accelerator achieves 994.74 GOPS of the throughput for U-Net. For general CNN implementation, the proposed scheme achieves 1080.9 GOPS when running VGG16 with 1.7 times less buffer size compared to previous works.