CT2: Colorization Transformer via Color Tokens

被引:21
作者
Weng, Shuchen [1 ]
Sun, Jimeng [2 ]
Li, Yu [3 ]
Li, Si [2 ]
Shi, Boxin [1 ]
机构
[1] Peking Univ, Sch Comp Sci, NERCVT, Beijing, Peoples R China
[2] Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing, Peoples R China
[3] Int Digital Econ Acad, Shenzhen, Peoples R China
来源
COMPUTER VISION, ECCV 2022, PT VII | 2022年 / 13667卷
基金
中国国家自然科学基金;
关键词
D O I
10.1007/978-3-031-20071-7_1
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Automatic image colorization is an ill-posed problem with multi-modal uncertainty, and there remains two main challenges with previous methods: incorrect semantic colors and under-saturation. In this paper, we propose an end-to-end transformer-based model to overcome these challenges. Benefited from the long-range context extraction of transformer and our holistic architecture, our method could colorize images with more diverse colors. Besides, we introduce color tokens into our approach and treat the colorization task as a classification problem, which increases the saturation of results. We also propose a series of modules to make image features interact with color tokens, and restrict the range of possible color candidates, which makes our results visually pleasing and reasonable. In addition, our method does not require any additional external priors, which ensures its well generalization capability. Extensive experiments and user studies demonstrate that our method achieves superior performance than previous works.
引用
收藏
页码:1 / 16
页数:16
相关论文
共 37 条
[1]  
[Anonymous], HUMAN VISION ELECT I
[2]  
Antic J., A deep learning based project for colorizing and restoring old images (and video!)
[3]  
Ardizzone L, 2019, ARXIV
[4]   Unsupervised Diverse Colorization via Generative Adversarial Networks [J].
Cao, Yun ;
Zhou, Zhiming ;
Zhang, Weinan ;
Yu, Yong .
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2017, PT I, 2017, 10534 :151-166
[5]  
Carion N., 2020, EUROPEAN C COMPUTER, V12346, P213, DOI 10.1007/978-3-030-58452-8_13
[6]   Pre-Trained Image Processing Transformer [J].
Chen, Hanting ;
Wang, Yunhe ;
Guo, Tianyu ;
Xu, Chang ;
Deng, Yiping ;
Liu, Zhenhua ;
Ma, Siwei ;
Xu, Chunjing ;
Xu, Chao ;
Gao, Wen .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :12294-12305
[7]   Deep Colorization [J].
Cheng, Zezhou ;
Yang, Qingxiong ;
Sheng, Bin .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :415-423
[8]  
Chu X., 2021, ARXIV, DOI DOI 10.48550/ARXIV.2102.10882
[9]   Learning Diverse Image Colorization [J].
Deshpande, Aditya ;
Lu, Jiajun ;
Yeh, Mao-Chuang ;
Chong, Min Jin ;
Forsyth, David .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2877-2885
[10]  
Dosovitskiy A., 2021, P INT C LEARN REPR, P11929