Successive Graph Convolutional Network for Image De-raining

被引:42
作者
Fu, Xueyang [1 ]
Qi, Qi [2 ]
Zha, Zheng-Jun [1 ]
Ding, Xinghao [2 ]
Wu, Feng [1 ]
Paisley, John [3 ,4 ]
机构
[1] Univ Sci & Technol China, Sch Informat Sci & Technol, Hefei, Peoples R China
[2] Xiamen Univ, Sch Informat, Xiamen, Peoples R China
[3] Columbia Univ, Dept Elect Engn, New York, NY 10027 USA
[4] Columbia Univ, Data Sci Inst, New York, NY USA
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
Image de-raining; Graph convolutional networks; Deep learning; Image processing; REMOVAL; VISION;
D O I
10.1007/s11263-020-01428-6
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Deep convolutional neural networks (CNNs) have shown their advantages in the single image de-raining task. However, most existing CNNs-based methods utilize only local spatial information without considering long-range contextual information. In this paper, we propose a graph convolutional networks (GCNs)-based model to solve the above problem. We specifically design two graphs to extract representations from new dimensions. The first graph models the global spatial relationship between pixels in the feature, while the second graph models the interrelationship across the channels. By integrating conventional CNNs and our GCNs into a single framework, the proposed method is able to explore comprehensive feature representations from three aspects, i.e., local spatial patterns, global spatial coherence and channel correlation. To better exploit the explored rich feature representations, we further introduce a simple yet effective recurrent operations to perform the de-raining process in a successive manner. Benefiting from the rich information exploration and exploitation, our method achieves state-of-the-art results on both synthetic and real-world data sets.
引用
收藏
页码:1691 / 1711
页数:21
相关论文
共 102 条
[1]  
Abadi Martin, 2016, arXiv
[2]   Contour Detection and Hierarchical Image Segmentation [J].
Arbelaez, Pablo ;
Maire, Michael ;
Fowlkes, Charless ;
Malik, Jitendra .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2011, 33 (05) :898-916
[3]   Analysis of Rain and Snow in Frequency Space [J].
Barnum, Peter C. ;
Narasimhan, Srinivasa ;
Kanade, Takeo .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2010, 86 (2-3) :256-274
[4]   Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks [J].
Bossu, Jeremie ;
Hautiere, Nicolas ;
Tarel, Jean-Philippe .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2011, 93 (03) :348-367
[5]   Transformed Low-rank Model for Line Pattern Noise Removal [J].
Chang, Yi ;
Yan, Luxin ;
Zhong, Sheng .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :1735-1743
[6]   Real-World Image Denoising with Deep Boosting [J].
Chen, Chang ;
Xiong, Zhiwei ;
Tian, Xinmei ;
Zha, Zheng-Jun ;
Wu, Feng .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020, 42 (12) :3071-3087
[7]   A Rain Pixel Recovery Algorithm for Videos With Highly Dynamic Scenes [J].
Chen, Jie ;
Chau, Lap-Pui .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2014, 23 (03) :1097-1104
[8]   Image Blind Denoising With Generative Adversarial Network Based Noise Modeling [J].
Chen, Jingwen ;
Chen, Jiawei ;
Chao, Hongyang ;
Yang, Ming .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3155-3164
[9]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[10]   A Generalized Low-Rank Appearance Model for Spatio-Temporally Correlated Rain Streaks [J].
Chen, Yi-Lei ;
Hsu, Chiou-Ting .
2013 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2013, :1968-1975