Human behaviour analysis based on spatio-temporal dual-stream heterogeneous convolutional neural network

At present, there are still many problems to be solved in human behaviour analysis, such as insufficient utilisation of behaviour characteristic information and slow operation rate. We propose a human behaviour analysis algorithm based on spatio-temporal dual-stream heterogeneous convolutional neural network (STDNet). The algorithm is improved on the basic structure of the traditional dual-stream network. When extracting spatial information, the DenseNet uses a hierarchical connection method to construct a dense network to extract the spatial feature of the video RGB image. When extracting motion information, BNInception is used to extract temporal features of video optical flow images. Finally, feature fusion is carried out by multi-layer perceptron and sent to Softmax classifier for classification. Experimental results on the UCF101 dataset show that the algorithm can effectively use the spatio-temporal feature information in video, reduce the amount of calculation of the network model, and greatly improve the ability to distinguish similar actions.