GNDAN: Graph Navigated Dual Attention Network for Zero-Shot Learning

被引:25
|
作者
Chen, Shiming [1 ]
Hong, Ziming [1 ]
Xie, Guosen [2 ]
Peng, Qinmu [1 ]
You, Xinge [1 ]
Ding, Weiping [3 ]
Shao, Ling [4 ]
机构
[1] Huazhong Univ Sci & Technol, Sch Elect Informat & Commun, Wuhan 430074, Peoples R China
[2] Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing, Peoples R China
[3] Nantong Univ, Sch Informat Sci & Technol, Nantong 226019, Peoples R China
[4] Saudi Data & Artificial Intelligence Author SDAIA, Natl Ctr Artificial Intelligence NCAI, Riyadh, Saudi Arabia
基金
中国国家自然科学基金;
关键词
Semantics; Visualization; Feature extraction; Task analysis; Knowledge transfer; Navigation; Learning systems; Attribute-based region features; graph attention network (GAT); graph neural network (GNN); zero-shot learning (ZSL);
D O I
10.1109/TNNLS.2022.3155602
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Zero-shot learning (ZSL) tackles the unseen class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a direct embedding is adopted for associating the visual and semantic domains in ZSL. However, most existing ZSL methods focus on learning the embedding from implicit global features or image regions to the semantic space. Thus, they fail to: 1) exploit the appearance relationship priors between various local regions in a single image, which corresponds to the semantic information and 2) learn cooperative global and local features jointly for discriminative feature representations. In this article, we propose the novel graph navigated dual attention network (GNDAN) for ZSL to address these drawbacks. GNDAN employs a region-guided attention network (RAN) and a region-guided graph attention network (RGAT) to jointly learn a discriminative local embedding and incorporate global context for exploiting explicit global embeddings under the guidance of a graph. Specifically, RAN uses soft spatial attention to discover discriminative regions for generating local embeddings. Meanwhile, RGAT employs an attribute-based attention to obtain attribute-based region features, where each attribute focuses on the most relevant image regions. Motivated by the graph neural network (GNN), which is beneficial for structural relationship representations, RGAT further leverages a graph attention network to exploit the relationships between the attribute-based region features for explicit global embedding representations. Based on the self-calibration mechanism, the joint visual embedding learned is matched with the semantic embedding to form the final prediction. Extensive experiments on three benchmark datasets demonstrate that the proposed GNDAN achieves superior performances to the state-of-the-art methods. Our code and trained models are available at https://github.com/shiming-chen/GNDAN.
引用
收藏
页码:4516 / 4529
页数:14
相关论文
共 50 条
  • [11] Language-Augmented Pixel Embedding for Generalized Zero-Shot Learning
    Wang, Ziyang
    Gou, Yunhao
    Li, Jingjing
    Zhu, Lei
    Shen, Heng Tao
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (03) : 1019 - 1030
  • [12] Semantic-Visual Combination Propagation Network for Zero-Shot Learning
    Song, Wenli
    Zhang, Lei
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2022, 69 (04) : 2341 - 2345
  • [13] Spherical Zero-Shot Learning
    Shen, Jiayi
    Xiao, Zehao
    Zhen, Xiantong
    Zhang, Lei
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (02) : 634 - 645
  • [14] Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder
    Xing, Nan
    Liu, Yang
    Zhu, Hong
    Wang, Jing
    Han, Jungong
    IEEE ACCESS, 2021, 9 : 733 - 742
  • [15] Learning MLatent Representations for Generalized Zero-Shot Learning
    Ye, Yalan
    Pan, Tongjie
    Luo, Tonghoujun
    Li, Jingjing
    Shen, Heng Tao
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 2252 - 2265
  • [16] Generalized Zero-Shot Learning With Multiple Graph Adaptive Generative Networks
    Xie, Guo-Sen
    Zhang, Zheng
    Liu, Guoshuai
    Zhu, Fan
    Liu, Li
    Shao, Ling
    Li, Xuelong
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (07) : 2903 - 2915
  • [17] Towards Unbiased Multi-Label Zero-Shot Learning With Pyramid and Semantic Attention
    Liu, Ziming
    Guo, Song
    Guo, Jingcai
    Xu, Yuanyuan
    Huo, Fushuo
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 7441 - 7455
  • [18] A Review of Generalized Zero-Shot Learning Methods
    Pourpanah, Farhad
    Abdar, Moloud
    Luo, Yuxuan
    Zhou, Xinlei
    Wang, Ran
    Lim, Chee Peng
    Wang, Xi-Zhao
    Wu, Q. M. Jonathan
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (04) : 4051 - 4070
  • [19] Generative Mixup Networks for Zero-Shot Learning
    Xu, Bingrong
    Zeng, Zhigang
    Lian, Cheng
    Ding, Zhengming
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022,
  • [20] Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning
    Shermin, Tasfia
    Teng, Shyh Wei
    Sohel, Ferdous
    Murshed, Manzur
    Lu, Guojun
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2022, 31 : 721 - 733