Cross-media similarity metric learning with unified deep networks

被引:0
作者
Jinwei Qi
Xin Huang
Yuxin Peng
机构
[1] Peking University,Institute of Computer Science and Technology
来源
Multimedia Tools and Applications | 2017年 / 76卷
关键词
Cross-media retrieval; Representation learning; Metric learning;
D O I
暂无
中图分类号
学科分类号
摘要
As a highlighting research topic in the multimedia area, cross-media retrieval aims to capture the complex correlations among multiple media types. Learning better shared representation and distance metric for multimedia data is important to boost the cross-media retrieval. Motivated by the strong ability of deep neural network in feature representation and comparison functions learning, we propose the Unified Network for Cross-media Similarity Metric (UNCSM) to associate cross-media shared representation learning with distance metric in a unified framework. First, we design a two-pathway deep network pretrained with contrastive loss, and employ double triplet similarity loss for fine-tuning to learn the shared representation for each media type by modeling the relative semantic similarity. Second, the metric network is designed for effectively calculating the cross-media similarity of the shared representation, by modeling the pairwise similar and dissimilar constraints. Compared to the existing methods which mostly ignore the dissimilar constraints and only use sample distance metric as Euclidean distance separately, our UNCSM approach unifies the representation learning and distance metric to preserve the relative similarity as well as embrace more complex similarity functions for further improving the cross-media retrieval accuracy. The experimental results show that our UNCSM approach outperforms 8 state-of-the-art methods on 4 widely-used cross-media datasets.
引用
收藏
页码:25109 / 25127
页数:18
相关论文
共 18 条
  • [1] Hardoon DR(2004)Canonical correlation analysis: An overview with application to learning methods Neural Comput 16 2639-2664
  • [2] Szedmák S(2006)A fast learning algorithm for deep belief nets Neural Comput 18 1527-1554
  • [3] Hinton GE(2012)Oracle in image search: A content-based approach to performance prediction ACM Trans Inf Syst 30 13:1-13:23
  • [4] Osindero S(2006)Clip-based similarity measure for query-dependent clip retrieval and video summarization IEEE Trans Circuits Syst Video Technol (TCSVT) 16 612-627
  • [5] Teh YW(2012)An efficient learning procedure for deep boltzmann machines Neural Comput 24 1967-2006
  • [6] Nie L(2008)Semantic subspace projection and its applications in image retrieval IEEE Trans Circuits Syst Video Technol (TCSVT) 18 544-548
  • [7] Wang M(2014)Learning cross-media joint representation with sparse and semi-supervised regularization IEEE Trans Circuits Syst Video Technol (TCSVT) 24 965-978
  • [8] Zha Z(undefined)undefined undefined undefined undefined-undefined
  • [9] Chua T(undefined)undefined undefined undefined undefined-undefined
  • [10] Peng Y(undefined)undefined undefined undefined undefined-undefined