Similarity Graph-correlation Reconstruction Network for unsupervised cross-modal hashing

被引:19
作者
Yao, Dan [1 ,2 ]
Li, Zhixin [1 ,2 ]
Li, Bo [1 ,2 ]
Zhang, Canlong [1 ,2 ]
Ma, Huifang [3 ]
机构
[1] Guangxi Normal Univ, Key Lab Educ Blockchain & Intelligent Technol, Minist Educ, Guilin 541004, Peoples R China
[2] Guangxi Normal Univ, Guangxi Key Lab Multisource Informat Min & Secur, Guilin 541004, Peoples R China
[3] Northwest Normal Univ, Coll Comp Sci & Engn, Lanzhou 730070, Peoples R China
基金
中国国家自然科学基金;
关键词
Cross-modal retrieval; Unsupervised cross-modal hashing; Similarity matrix; Graph rebasing; Similarity reconstruction;
D O I
10.1016/j.eswa.2023.121516
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Existing cross-modal hash retrieval methods can simultaneously enhance retrieval speed and reduce storage space. However, these methods face a major challenge in determining the similarity metric between two modalities. Specifically, the accuracy of intra-modal and inter-modal similarity measurements is inadequate, and the large gap between modalities leads to semantic bias. In this paper, we propose a Similarity Graph-correlation Reconstruction Network (SGRN) for unsupervised cross-modal hashing. Particularly, the local relation graph rebasing module is used to filter out graph nodes with weak similarity and associate graph nodes with strong similarity, resulting in fine-grained intra-modal similarity relation graphs. The global relation graph reconstruction module is further strengthens cross-modal correlation and implements fine-grained similarity alignment between modalities. In addition, in order to bridge the modal gap, we combine the similarity representation of real-valued and hash features to design the intra-modal and inter-modal training strategies. SGRN conducted extensive experiments on two cross-modal retrieval datasets, and the experimental results effectively validated the superiority of the proposed method and significantly improved the retrieval performance.
引用
收藏
页数:13
相关论文
共 54 条
[1]   HashNet: Deep Learning to Hash by Continuation [J].
Cao, Zhangjie ;
Long, Mingsheng ;
Wang, Jianmin ;
Yu, Philip S. .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :5609-5618
[2]   Self-attention and adversary learning deep hashing network for cross-modal retrieval [J].
Chen, Shubai ;
Wu, Song ;
Wang, Li ;
Yu, Zhenyang .
COMPUTERS & ELECTRICAL ENGINEERING, 2021, 93
[3]   Hierarchical semantic interaction-based deep hashing network for cross-modal retrieval [J].
Chen, Shubai ;
Wu, Song ;
Wang, Li .
PEERJ COMPUTER SCIENCE, 2021,
[4]   Deep Semantic-Preserving Reconstruction Hashing for Unsupervised Cross-Modal Retrieval [J].
Cheng, Shuli ;
Wang, Liejun ;
Du, Anyu .
ENTROPY, 2020, 22 (11) :1-22
[5]  
Chua T.-S., 2009, P ACM INT C IM VID R
[6]   Probabilistic Embeddings for Cross-Modal Retrieval [J].
Chun, Sanghyuk ;
Oh, Seong Joon ;
de Rezende, Rafael Sampaio ;
Kalantidis, Yannis ;
Larlus, Diane .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :8411-8420
[7]   On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval [J].
Costa Pereira, Jose ;
Coviello, Emanuele ;
Doyle, Gabriel ;
Rasiwasia, Nikhil ;
Lanckriet, Gert R. G. ;
Levy, Roger ;
Vasconcelos, Nuno .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2014, 36 (03) :521-535
[8]  
Dejie Yang, 2020, ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval, P44, DOI 10.1145/3372278.3390673
[9]   Discrete matrix factorization hashing for cross-modal retrieval [J].
Fang, Xiaozhao ;
Liu, Zhihu ;
Han, Na ;
Jiang, Lin ;
Teng, Shaohua .
INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2021, 12 (10) :3023-3036
[10]   Average Approximate Hashing-Based Double Projections Learning for Cross-Modal Retrieval [J].
Fang, Xiaozhao ;
Jiang, Kaihang ;
Han, Na ;
Teng, Shaohua ;
Zhou, Guoxu ;
Xie, Shengli .
IEEE TRANSACTIONS ON CYBERNETICS, 2022, 52 (11) :11780-11793