FPGA based convolution and memory architecture for Convolutional Neural Network

Convolutional Neural Networks(CNNs) are widely used in vision based applications to increase the performance but at the cost of higher storage and increase in computation. Hardware implementations of CNN are limited by the computational complexity and bandwidth while accessing off-chip memory. In this work a novel FPGA based hardware architecture for 2D convolution operation with reduced computational complexity using Winograd's 2D minimal filtering algorithm and a memory architecture to reduce on-chip read operations to access adjacent input data tiles for convolution operations is proposed to accelerate CNNs. An on-chip memory bank reuse architecture is also utilized to reduce the number of memory read and write operations to off-chip memory. The proposed architecture for convolution operation achieves lower computational complexity by reducing the number of multiplication operations without proportionate increase in number of addition operations compared to prior implementations. The number of data read operations from on-chip memory is reduced by 4 times and using the on-chip memory bank reuse scheme latency associated with accessing intermediate data is reduced. The implemented uses 16-bit fixed point representation which could reduce bit width to save area and energy. Virtex Ultra scale+ VCU118 Evaluation Board 2.0 populated with XCVU9P-L2FLGA2104 is used as the platform for implementing the design. VGG Net based CNN is used for the implementation. The computation time for individual convolutional layer is also estimated and it is found to be reduced. For a 3x3 kernel the number of multiplications is reduced to 4 from 9 compared to standard convolution operation and the number of addition operations reduced to 12 from 14 compared to prior hardware implementations of Winograd's 2D minimal filtering algorithm.