Improved human-object interaction detection through skeleton-object relations

Current methods for human-object interaction detection often use the spatial relation between a human and an object as an interaction pattern. However, this strategy is relatively simple and has low discrimination in similar interactions. To solve this drawback, the spatial relation between skeletons and objects is proposed to model the interaction pattern and improve the detection accuracy. First, the skeleton-object interaction pattern image is extracted for each interaction proposal. Second, a deep neural network is applied to learn the interaction features from these images. Finally, the interaction feature is added to the human-object interaction detection network by a multistream structure. In the experiments, we evaluate the proposed method on the HICO-DET and V-COCO datasets. Experimental results show that the proposed method can achieve the best performance compared with state-of-art methods.