LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

被引:0
|
作者
Ma, Xingjiang [1 ]
Yang, Jing [1 ,2 ]
Lin, Jiacheng [3 ]
Zheng, Zhenzhe [4 ]
Li, Shaobo [1 ]
Hu, Bingqi [1 ]
Tang, Xianghong [1 ]
机构
[1] Guizhou Univ, State Key Lab Publ Big Data, Guiyang 550025, Peoples R China
[2] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[3] Hunan Univ, Coll Comp Sci & Elect Engn, Changsha 410082, Peoples R China
[4] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
基金
中国国家自然科学基金;
关键词
Visualization; Feature extraction; Dogs; Task analysis; Attention mechanisms; Zero-shot learning; Circuits and systems; Compositional zero-shot learning; visual attributes; objects and attributes; inter-class connectivity; OBJECTS;
D O I
10.1109/TCSVT.2024.3444782
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Compositional Zero-Shot Learning (CZSL) has been applied to various scenarios, including scene understanding, visual-language representation, and domain adaptation. Despite numerous endeavours and significant advancements, the crucial issues of fuzzy conceptualization of visual attributes and insufficient inter-class connectivity, have remained insufficiently addressed. To address these issues, we propose Learning Visual Attributes Representation for Compositional Zero-Shot Learning (LVAR-CZSL), which has the ability to learn visual attributes and inter-class dependencies. LVAR-CZSL is mainly composed of two key components: the Visual Attribute Representation Module (VARM) and the Connected Learning Module (CLM). Specifically, VARM extracts detailed attributes and object visual features from global visual features, resolving the issue of fuzzy visual attribute concepts. Moreover, CLM endows LVAR-CZSL with the capability to perceive connectivity between different attributes and objects, effectively enhancing inter-class connectivity. To establish a close connection between VARM and CLM and minimize the gap between image and text features, we introduce the composition-attribute-object Joint Scoring Function (JSF). Additionally, we propose Joint Loss Function (JLF) to optimize the learning process of VARM and CLM. The experiment results on four datasets show that LVAR-CZSL achieves state-of-the-art performance. The code is available at https://github.com/mxjmxj1/LVAR-CZSL.
引用
收藏
页码:13311 / 13323
页数:13
相关论文
共 50 条
  • [31] Knowledge Guided Transformer Network for Compositional Zero-Shot Learning
    Panda, Aditya
    Prasad, Dipti
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (11)
  • [32] Generative Mixup Networks for Zero-Shot Learning
    Xu, Bingrong
    Zeng, Zhigang
    Lian, Cheng
    Ding, Zhengming
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022,
  • [33] Swap-Reconstruction Autoencoder for Compositional Zero-Shot Learning
    Guo, Ting
    Liang, Jiye
    Xie, Guo-Sen
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 438 - 443
  • [34] Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learning
    Li, Wenrui
    Wang, Penghong
    Xiong, Ruiqin
    Fan, Xiaopeng
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2024, 33 : 4840 - 4852
  • [35] Grouping attributes zero-shot learning for tongue constitution recognition
    Wen, Guihua
    Ma, Jiajiong
    Hu, Yang
    Li, Huihui
    Jiang, Lijun
    ARTIFICIAL INTELLIGENCE IN MEDICINE, 2020, 109
  • [36] Zero-Shot Learning with Missing Attributes using Semantic Correlations
    Braytee, Ali
    Naji, Mohamad
    Anaissi, Ali
    Chaturvedi, Kunal
    Prasad, Mukesh
    2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [37] Joint Visual and Semantic Optimization for zero-shot learning
    Wu, Hanrui
    Yan, Yuguang
    Chen, Sentao
    Huang, Xiangkang
    Wu, Qingyao
    Ng, Michael K.
    KNOWLEDGE-BASED SYSTEMS, 2021, 215 (215)
  • [38] Research progress of zero-shot learning
    Sun, Xiaohong
    Gu, Jinan
    Sun, Hongying
    APPLIED INTELLIGENCE, 2021, 51 (06) : 3600 - 3614
  • [39] Research progress of zero-shot learning
    Xiaohong Sun
    Jinan Gu
    Hongying Sun
    Applied Intelligence, 2021, 51 : 3600 - 3614
  • [40] Zero-Shot Learning With Transferred Samples
    Guo, Yuchen
    Ding, Guiguang
    Han, Jungong
    Gao, Yue
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2017, 26 (07) : 3277 - 3290