Full-view salient feature mining and alignment for text-based person search

被引:4
|
作者
Xie, Sheng [1 ]
Zhang, Canlong [1 ,2 ]
Ning, Enhao [1 ]
Li, Zhixin [1 ,2 ]
Wang, Zhiwen [3 ]
Wei, Chunrong [4 ]
机构
[1] Guangxi Normal Univ, Key Lab Educ Blockchain & Intelligent Technol, Minist Educ, Guilin 541004, Peoples R China
[2] Guangxi Normal Univ, Guangxi Key Lab Multisource Informat Min & Secur, Guilin 541004, Peoples R China
[3] Guangxi Univ Sci & Technol, Sch Comp Sci & Technol, Liuzhou 545006, Peoples R China
[4] Guangxi Normal Univ, Teachers Coll Vocat & Tech Educ, Guilin 541004, Peoples R China
基金
中国国家自然科学基金;
关键词
Text-based person search; Diffusion; Full-view; Generation; Text attention; OPTIMIZATION; NETWORK;
D O I
10.1016/j.eswa.2024.124071
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Text-based person search aims to retrieve relevant person images from a large database given textual queries. However, single-view limitation of surveillance cameras and cross-modal heterogeneity still remain challenging open issues. To address these, we propose a F ul l -view S a lient Feature Mining N etwork (FLAN) to improve text-image matching in this task. Our FLAN introduces two key innovations. First, the Diffusion-based Fullview Image Augmentation generates informative full-view data from a single image to simulate human visual observation and learn view-invariant features. Second, the Dual-max Text Attention module optimizes spatial and channel-wise text attentions to extract the most discriminative words characterizing the person. Together, these innovations handle insufficient, imbalanced, and heterogeneous data for more accurate matching. Extensive experiments on three text-based person search datasets, CUHK-PEDES, ICFG-PEDES and RSTPReid, demonstrate superior performance of our FLAN with improved robustness and generalization.
引用
收藏
页数:13
相关论文
共 44 条
  • [1] Feature semantic alignment and information supplement for Text-based person search
    Zhou, Hang
    Li, Fan
    Tian, Xuening
    Huang, Yuling
    FRONTIERS IN PHYSICS, 2023, 11
  • [2] Joint Token and Feature Alignment Framework for Text-Based Person Search
    Li, Shangze
    Lu, Andong
    Huang, Yan
    Li, Chenglong
    Wang, Liang
    IEEE SIGNAL PROCESSING LETTERS, 2022, 29 : 2238 - 2242
  • [3] Conditional Feature Learning Based Transformer for Text-Based Person Search
    Gao, Chenyang
    Cai, Guanyu
    Jiang, Xinyang
    Zheng, Feng
    Zhang, Jun
    Gong, Yifei
    Lin, Fangzhou
    Sun, Xing
    Bai, Xiang
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2022, 31 : 6097 - 6108
  • [4] Asymmetric Cross-Scale Alignment for Text-Based Person Search
    Ji, Zhong
    Hu, Junhua
    Liu, Deyin
    Wu, Lin Yuanbo
    Zhao, Ye
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 7699 - 7709
  • [5] Text-Guided Visual Feature Refinement for Text-Based Person Search
    Gao, Liying
    Niu, Kai
    Ma, Zehong
    Jiao, Bingliang
    Tan, Tonghao
    Wang, Peng
    PROCEEDINGS OF THE 2021 INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL (ICMR '21), 2021, : 118 - 126
  • [6] Cross-modal alignment with synthetic caption for text-based person search
    Zhao, Weichen
    Lu, Yuxing
    Liu, Zhiyuan
    Yang, Yuan
    Jiao, Ge
    INTERNATIONAL JOURNAL OF MULTIMEDIA INFORMATION RETRIEVAL, 2025, 14 (02)
  • [7] CLIP-Based Multi-level Alignment for Text-based Person Search
    Wu, Zhijun
    Ma, Shiwei
    2024 5TH INTERNATIONAL CONFERENCE ON COMPUTER ENGINEERING AND APPLICATION, ICCEA 2024, 2024, : 610 - 614
  • [8] LEARNING SEMANTIC-ALIGNED FEATURE REPRESENTATION FOR TEXT-BASED PERSON SEARCH
    Li, Shiping
    Cao, Min
    Zhang, Min
    2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 2724 - 2728
  • [9] Text-based Person Search in Full Images via Semantic-Driven Proposal Generation
    Zhang, Shizhou
    Cheng, De
    Luo, Wenlong
    Xing, Yinghui
    Long, Duo
    Li, Hao
    Niu, Kai
    Liang, Guoqiang
    Zhang, Yanning
    PROCEEDINGS OF THE 4TH INTERNATIONAL WORKSHOP ON HUMAN-CENTRIC MULTIMEDIA ANALYSIS, HCMA 2023, 2023, : 5 - 14
  • [10] Fine-grained semantic oriented embedding set alignment for text-based person search
    Zhao, Jiaqi
    Fu, Ao
    Zhou, Yong
    Du, Wen-liang
    Yao, Rui
    IMAGE AND VISION COMPUTING, 2024, 152