FA-IATI: A Framework of Frequency Adaptive and Iterative Attention Interaction for Image-Text Matching

The matching relationship between language and vision, which extensively involves various fields such as search engines and social media, is a hot topic that researchers are exploring. Existing matching methods pay more attention to alignment of features and lack the reasoning of high-level semantic concepts, especially the difference in visual expression, inside the modal. Therefore, we propose a frequency adaptive and iterative attention interaction for image-text matching (FA-IATI) framework, starting from the perspective of capturing visual semantic relationships. Specifically, we adaptively aggregate low-frequency and high-frequency signals by using graph convolutional networks to enhance the contextual information between image regions. An attention interaction module generates global features through an iterative mechanism and gradually achieves semantic alignment during the aggregation of words and image regions. Experiments show that our FA-IATI model achieves the best results of 98.4% (R@10) and 94.9% (R@10) on the MS COCO dataset (using 1K testing) compared with the baseline model on text query and image query, respectively. Compared with other current advanced matching models, FA-IATI has superior performance and strong competitiveness.