Hybrid cross-domain joint network for sketch-based image retrieval

被引：0

作者：

Li Q. ^{[1
]}

Zhou Y. ^{[2
]}

Li C. ^{[2
]}

Peng Y. ^{[2
]}

Liang X. ^{[1
]}

机构：

[1] Tenth Institute of China Electronics Technology Group Corporation, Chengdu

[2] School of Electrical and Information Engineering, Tianjin University, Tianjin

来源：

Harbin Gongye Daxue Xuebao/Journal of Harbin Institute of Technology | 2022年 / 54卷 / 05期

关键词：

Cross-modal; Image retrieval; Neural networks; Sketch retrieval;

D O I：

10.11918/202108065

中图分类号：

学科分类号：

摘要：

Sketch-based cross-domain image retrieval (SBIR) uses a sketch as query to retrieve the most similar image from the color image database. In this study, in order to better fuse the features from sketch and color image, a hybrid cross-domain joint network for sketch-based image retrieval was proposed, consisting of a sketch feature extraction branch and a color image heterogeneous feature fusion network branch. The network extracts the feature representations of sketch, positive and negative color image, and corresponding edge outline, and fuses the features of the color image and its sketch approximation (the edge outline of the color image) as the color image feature, which bridges the cross-domain gap between sketches and images. The network model parameters and network structure were further explored to optimize the purposed algorithm. Experiment on Flickr15K dataset shows that the proposed method performed better than other advanced image retrieval methods. The mean average retrieval accuracy of the proposed method was 0.584 8, which was 0.052 2 higher than the optimal value in other methods. Copyright ©2022 Journal of Harbin Institute of Technology.All rights reserved.

引用

页码：64 / 73

页数：9

共 52 条

[31]

KRIZHEVSKY A, SUTSKEVER I, HINTON G E., ImageNet classification with deep convolutional neural networks, Proceedings of the Advances in Neural Information Processing Systems, (2012)

[32]

QI Yonggang, SONG Yizhe, ZHANG Honggang, Et al., Sketch-based image retrieval via Siamese convolutional neural network, Proceedings of the 2016 IEEE International Conference on Image Processing, (2016)

[33]

YU Qian, LIU Feng, SONG Yizhe, Et al., Sketch me that shoe, Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, (2016)

[34]

BUI T, RIBEIRO L, PONTI M, Et al., Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network, Computer Vision and Image Understanding, 164, (2017)

[35]

SANGKLOY P, BURNELL N, HAM C, Et al., The sketchy database: Learning to retrieve badly drawn bunnies, ACM Transactions on Graphics, 35, 4, (2016)

[36]

BUI T, RIBEIRO L S, PONTI M A, Et al., Generalisation and sharing in triplet convnets for sketch based visual search

[37]

KUMAR M P, PACKER B, KOLLER D., Self-paced learning for latent variable models, Proceedings of the 23rd International Conference on Neural Information Processing Systems, (2011)

[38]

BENGIO Y, LOURADOUR J, COLLOBERTR, Et al., Curriculum learning, Proceedings of the 26th Annual International Conference on Machine Learning, (2009)

[39]

XU Dan, ALAMEDA-PINEDA X, SONG Jingkuan, Et al., Cross-paced representation learning with partial curricula for sketch-based image retrieval, IEEE Transactions on Image Processing, 27, 9, (2018)

[40]

ZHOU Yuan, FENG Liyang, HOU Chunping, Et al., Hyperspectral and multispectral image fusion based on local low rank and coupled spectral unmixing, IEEE Transactions on Geoscience and Remote Sensing, 55, 10, (2017)

← 1 2 3 4 5 6 →