HOI as Embeddings: Advancements of Model Representation Capability in Human-Object Interaction Detection

In recent years, Human-Object Interaction Detection (HOID) has attracted increasing attention in the computer vision community and has been greatly advanced by the introduction of transformer-based models. However, the representation capability of the pre-trained object detection model is insufficient for capturing the complex interactions between humans and objects, which limits the performance of HOID methods. In this paper, we introduce three methods to progressively enhance the representation capability. (1) We propose QAHOI to take advantage of multi-scale feature maps with different spatial scales. (2) We propose PQNet to speed up training convergence with parallel queries. (3) We propose SOV-STG to combine the merits of QAHOI and PQNet and introduce the denoising learning strategy to further improve training convergence and performance. Our proposed method SOV-STG achieves state-of-the-art performance on the HICO-DET dataset with one-third of the training epochs compared to previous SOTA methods.