Enhancing Sketch-Based Image Retrieval by CNN Semantic Re-ranking

被引:98
|
作者
Wang, Luo [1 ,2 ]
Qian, Xueming [1 ,3 ]
Zhang, Yuting [1 ,2 ]
Shen, Jialie [4 ]
Cao, Xiaochun [5 ]
机构
[1] Xi An Jiao Tong Univ, SMILES Lab, Xian 710049, Peoples R China
[2] Xi An Jiao Tong Univ, Sch Elect & Informat Engn, Xian 710049, Peoples R China
[3] Xi An Jiao Tong Univ, Minist Educ, Key Lab Intelligent Networks & Network Secur, Xian 710049, Peoples R China
[4] Queens Univ Belfast, Sch Elect Elect Engn & Comp Sci, Belfast BT7 1NN, Antrim, North Ireland
[5] Chinese Acad Sci, Inst Informat Engn, Beijing 100093, Peoples R China
基金
中国国家自然科学基金;
关键词
Semantics; Feature extraction; Image edge detection; Image retrieval; Bridges; Noise measurement; Data mining; Classification; convolutional neural network (CNN); re-ranking; sketch-based image retrieval (SBIR);
D O I
10.1109/TCYB.2019.2894498
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper introduces a convolutional neural network (CNN) semantic re-ranking system to enhance the performance of sketch-based image retrieval (SBIR). Distinguished from the existing approaches, the proposed system can leverage category information brought by CNNs to support effective similarity measurement between the images. To achieve effective classification of query sketches and high-quality initial retrieval results, one CNN model is trained for classification of sketches, another for that of natural images. Through training dual CNN models, the semantic information of both the sketches and natural images is captured by deep learning. In order to measure the category similarity between images, a category similarity measurement method is proposed. Category information is then used for re-ranking. Re-ranking operation first infers the retrieval category of the query sketch and then uses the category similarity measurement to measure the category similarity between the query sketch and each initial retrieval result. Finally, the initial retrieval results are re-ranked. The experiments on different types of SBIR datasets demonstrate the effectiveness of the proposed re-ranking method. Comparisons with other re-ranking algorithms are also given to show the proposed method's superiority. Further, compared to the baseline systems, the proposed re-ranking approach achieves significantly higher precision in the top ten different SBIR methods and datasets.
引用
收藏
页码:3330 / 3342
页数:13
相关论文
共 50 条
  • [31] Learnable Pillar-based Re-ranking for Image-Text Retrieval
    Qu, Leigang
    Liu, Meng
    Wang, Wenjie
    Zheng, Zhedong
    Nie, Liqiang
    Chua, Tat-Seng
    PROCEEDINGS OF THE 46TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2023, 2023, : 1252 - 1261
  • [32] RefinerHash: a new hashing-based re-ranking technique for image retrieval
    Sabahi, Farzad
    Ahmad, M. Omair
    Swamy, M. N. S.
    MULTIMEDIA SYSTEMS, 2024, 30 (03)
  • [33] Feature Fusion-Based Re-Ranking for Home Textile Image Retrieval
    Miao, Ziyi
    Yao, Lan
    Zeng, Feng
    Wang, Yi
    Hong, Zhiguo
    MATHEMATICS, 2024, 12 (14)
  • [34] Object-based Image Retrieval with Attention Analysis and Spatial Re-ranking
    Gao, Ke
    Lin, Shouxun
    Zhang, Yongdong
    Tang, Sheng
    INTELLIGENT INFORMATION PROCESSING IV, 2008, : 118 - +
  • [35] Combining re-ranking and rank aggregation methods for image retrieval
    Guimaraes Pedronette, Daniel Carlos
    Torres, Ricardo da S.
    MULTIMEDIA TOOLS AND APPLICATIONS, 2016, 75 (15) : 9121 - 9144
  • [36] Combining re-ranking and rank aggregation methods for image retrieval
    Daniel Carlos Guimarães Pedronette
    Ricardo da S. Torres
    Multimedia Tools and Applications, 2016, 75 : 9121 - 9144
  • [37] MindCamera: Interactive Sketch-Based Image Retrieval and Synthesis
    Wang, Jingyu
    Zhao, Yu
    Qi, Qi
    Huo, Qiming
    Zou, Jian
    Ge, Ce
    Liao, Jianxin
    IEEE ACCESS, 2018, 6 : 3765 - 3773
  • [38] Sketch-based Shadow Image Retrieval for Digital Library
    Song, Jinjoo
    Cho, Hyeyoun
    Yoon, Sang Min
    2017 INTERNATIONAL CONFERENCE ON CULTURE AND COMPUTING (CULTURE AND COMPUTING), 2017, : 127 - 128
  • [39] Sketch-based Image Retrieval via Shape Words
    Xiao, Changcheng
    Wang, Changhu
    Zhang, Liqing
    Zhang, Lei
    ICMR'15: PROCEEDINGS OF THE 2015 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, 2015, : 571 - 574
  • [40] BHoG: binary descriptor for sketch-based image retrieval
    Haiyan Fu
    Hanguang Zhao
    Xiangwei Kong
    Xianbo Zhang
    Multimedia Systems, 2016, 22 : 127 - 136