Hybrid cross-domain joint network for sketch-based image retrieval

被引：0
作者：
Li Q. [1 ]
Zhou Y. [2 ]
Li C. [2 ]
Peng Y. [2 ]
Liang X. [1 ]
机构：
[1] Tenth Institute of China Electronics Technology Group Corporation, Chengdu
[2] School of Electrical and Information Engineering, Tianjin University, Tianjin
来源：
Harbin Gongye Daxue Xuebao/Journal of Harbin Institute of Technology | 2022年 / 54卷 / 05期
关键词：
Cross-modal; Image retrieval; Neural networks; Sketch retrieval;
D O I：
10.11918/202108065
中图分类号：
学科分类号：
摘要：
Sketch-based cross-domain image retrieval (SBIR) uses a sketch as query to retrieve the most similar image from the color image database. In this study, in order to better fuse the features from sketch and color image, a hybrid cross-domain joint network for sketch-based image retrieval was proposed, consisting of a sketch feature extraction branch and a color image heterogeneous feature fusion network branch. The network extracts the feature representations of sketch, positive and negative color image, and corresponding edge outline, and fuses the features of the color image and its sketch approximation (the edge outline of the color image) as the color image feature, which bridges the cross-domain gap between sketches and images. The network model parameters and network structure were further explored to optimize the purposed algorithm. Experiment on Flickr15K dataset shows that the proposed method performed better than other advanced image retrieval methods. The mean average retrieval accuracy of the proposed method was 0.584 8, which was 0.052 2 higher than the optimal value in other methods. Copyright ©2022 Journal of Harbin Institute of Technology.All rights reserved.
引用
收藏
页码：64 / 73
页数：9
相关论文
共 52 条
  • [1] ZHOU Yuan, WANG Ruolin, LI Hongru, Et al., Temporal action localization using long short-term dependency, IEEE Transactions on Multimedia, (2020)
  • [2] XIE Hongtao, GAO Ke, ZHANG Yongdong, Et al., Efficient feature detection and effective post-verification for large scale near-duplicate image search, IEEE Transactions on Multimedia, 13, 6, (2011)
  • [3] ZHOU Rong, CHEN Liuli, ZHANG Liqing, Sketch-based image retrieval on a large scale database, Proceedings of the 20th ACM international conference on Multimedia, (2012)
  • [4] CHU Lingyang, JIANG Shuqiang, WANG Shuhui, Et al., Robust spatial consistency graph model for partial duplicate image retrieval, IEEE Transactions on Multimedia, 15, 8, (2013)
  • [5] PARUI S, MITTAL A., Similarity-invariant sketch-based image retrieval in large databases, Proceedings of the European Conference on Computer Vision, (2014)
  • [6] HE Ran, ZHANG Man, WANG Liang, Et al., Cross-modal learning subspace learning via pairwise constraints, IEEE Transactions on Image Processing, 24, 12, (2015)
  • [7] XU Zhongwen, YANG Yi, KASSIM A, Et al., Cross-media relevance mining for evaluating text-based image search engine, Proceedings of the 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), (2014)
  • [8] WU Dayan, LIU Jing, LI Bo, Et al., Deep index-compatible hashing for fast image retrieval, Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), (2018)
  • [9] TANG Jinya, ZHANG Dongming, ZHANG Yongdong, Et al., Region similarity arrangement for image retrieval, Proceedings of the 2016 IEEE International Conference on Multimedia and Expo (ICME), (2016)
  • [10] DOU Zhi, WANG Ning, LI Baopu, Et al., Dual color space guided sketch colorization, IEEE Transactions on Image Processing, 30, (2021)
123456