COMatchNet: Co-Attention Matching Network for Video Object Segmentation

被引:1
|
作者
Huang, Lufei [1 ]
Sun, Fengming [1 ]
Yuan, Xia [1 ]
机构
[1] Nanjing Univ Sci & Technol, Nanjing 210094, Peoples R China
来源
PATTERN RECOGNITION, ACPR 2021, PT I | 2022年 / 13188卷
关键词
Co-attention; Pixel-level matching; Video object segmentation;
D O I
10.1007/978-3-031-02375-0_20
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Semi-supervised video object segmentation (semi-VOS) predicts pixel-accurate masks of the target objects in all frames according to the ground truth mask provided in the first frame. A critical challenge to this task is how to model the dependency between the query frame and other frames. Most methods neglect or do not make full use of the inherent relevance. In this paper, we propose a novel network called CO-Attention Matching Network (COMatchNet) for semi-VOS. The COMatchNet mainly consists of a co-attention module and a matching module. The co-attention module extracts frame correlation among the query frame and the previous frame and the first frame. The matching module calculates pixel-level matching scores and finds the most similar regions to preceding frames in the query frame. The COMatchNet integrates two level information and generates fine-grained segmentation masks. We conduct extensive experiments on three popular video object segmentation benchmarks, i.e. DAVIS 2016; DAVIS 2017; YouTube-VOS. Our COMatchNet achieves competitive performance (J&F) of 86.8%, 75.9%, and 81.4% on the above benchmarks, respectively.
引用
收藏
页码:271 / 284
页数:14
相关论文
共 50 条
  • [1] Co-attention Propagation Network for Zero-Shot Video Object Segmentation
    Pei, Gensheng
    Yao, Yazhou
    Shen, Fumin
    Huang, Dan
    Huang, Xingguo
    Shen, Heng-Tao
    arXiv, 2023,
  • [2] Hierarchical Co-Attention Propagation Network for Zero-Shot Video Object Segmentation
    Pei, Gensheng
    Yao, Yazhou
    Shen, Fumin
    Huang, Dan
    Huang, Xingguo
    Shen, Heng-Tao
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 : 2348 - 2359
  • [3] Zero-Shot Video Object Segmentation With Co-Attention Siamese Networks
    Lu, Xiankai
    Wang, Wenguan
    Shen, Jianbing
    Crandall, David
    Luo, Jiebo
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (04) : 2228 - 2242
  • [4] Co-attention CNNs for Unsupervised Object Co-segmentation
    Hsu, Kuang-Jui
    Lin, Yen-Yu
    Chuang, Yung-Yu
    PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2018, : 748 - 756
  • [5] Learning Motion-Appearance Co-Attention for Zero-Shot Video Object Segmentation
    Yang, Shu
    Zhang, Lu
    Qi, Jinqing
    Lu, Huchuan
    Wang, Shuo
    Zhang, Xiaoxing
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 1544 - 1553
  • [6] See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks
    Lu, Xiankai
    Wang, Wenguan
    Ma, Chao
    Shen, Jianbing
    Shao, Ling
    Porikli, Fatih
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 3618 - 3627
  • [7] Group-wise Deep Object Co-Segmentation with Co-Attention Recurrent Neural Network
    Li, Bo
    Sun, Zhengxing
    Li, Qian
    Wu, Yunjie
    Hu, Anqi
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 8518 - 8527
  • [8] Co-Attention for Conditioned Image Matching
    Wiles, Olivia
    Ehrhardt, Sebastien
    Zisserman, Andrew
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 15915 - 15924
  • [9] Streamer temporal action detection in live video by co-attention boundary matching
    Chenhao Li
    Chen He
    Hui Zhang
    Jiacheng Yao
    Jing Zhang
    Li Zhuo
    International Journal of Machine Learning and Cybernetics, 2022, 13 : 3071 - 3088
  • [10] Hierarchical Memory Matching Network for Video Object Segmentation
    Seong, Hongje
    Oh, Seoung Wug
    Lee, Joon-Young
    Lee, Seongwon
    Lee, Suhyeon
    Kim, Euntai
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 12869 - 12878