DHTCUN: Deep Hybrid Transformer CNN U Network for Single-Image Super-Resolution

被引:8
作者
Talreja, Jagrati [1 ]
Aramvith, Supavadee [2 ]
Onoye, Takao [3 ]
机构
[1] Chulalongkorn Univ, Fac Engn, Dept Elect Engn, Bangkok 10330, Thailand
[2] Chulalongkorn Univ, Fac Engn, Dept Elect Engn, Multimedia Data Analyt & Proc Unit, Bangkok 10330, Thailand
[3] Osaka Univ, Grad Sch Informat Sci & Technol, Suita, Osaka 5650871, Japan
关键词
CNN; enhanced spatial attention; single-image super-resolution; Transformer;
D O I
10.1109/ACCESS.2024.3450300
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recent advances in image super-resolution have investigated various transformer and CNN techniques to improve quantitative and perceptual outcomes. Reconstructing high-resolution images from their low-resolution equivalents by combining the power of transformers and CNN has been a crucial task in recent times. We propose a novel U-shaped architecture that integrates transformers and convolutional neural networks (CNNs) to leverage the strengths of both approaches. The network incorporates a novel Parallel Hybrid Transformer CNN Block (PHTCB) on the backbone of the U-shaped design, ensuring computational efficiency and robust hierarchical feature representation. Our architecture incorporates triple-enhanced spatial-attention mechanisms and a Transformer CNN (TCN) Block in PHTCB. The TCN Block helps preserve sharp edges and intricate details often lost in traditional SISR methods and enhances the visual fidelity of the reconstructed high-resolution images. Additionally, we introduce the triple-enhanced spatial attention (TESA) approach that helps precisely localize of important features. Blurring can be reduced for crucial features by focusing on these critical areas because of the network's ability to control features at various scales. Experiments demonstrate that our proposed method yields better quantitative measurements, including visually appealing high-resolution image reconstructions, peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM).
引用
收藏
页码:122624 / 122641
页数:18
相关论文
共 12 条
[1]  
Ai WJ, 2020, IEEE IMAGE PROC, P1586, DOI [10.1109/ICIP40778.2020.9191314, 10.1109/icip40778.2020.9191314]
[2]   Denoising convolutional neural network with mask for salt and pepper noise [J].
Chen, Jiuning ;
Li, Fang .
IET IMAGE PROCESSING, 2019, 13 (13) :2604-2613
[3]   Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space [J].
Dabov, Kostadin ;
Foi, Alessandro ;
Katkovnik, Vladimir ;
Egiazarian, Karen .
2007 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-7, 2007, :313-316
[4]   Accelerating the Super-Resolution Convolutional Neural Network [J].
Dong, Chao ;
Loy, Chen Change ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2016, PT II, 2016, 9906 :391-407
[5]   Image Super-Resolution Using Deep Convolutional Networks [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (02) :295-307
[6]   Super-Resolution in Medical Imaging [J].
Greenspan, Hayit .
COMPUTER JOURNAL, 2009, 52 (01) :43-63
[7]   Weighted Nuclear Norm Minimization with Application to Image Denoising [J].
Gu, Shuhang ;
Zhang, Lei ;
Zuo, Wangmeng ;
Feng, Xiangchu .
2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2014, :2862-2869
[8]  
Malczewski K, 2009, STUD COMPUT INTELL, V231, P171
[9]   Sketch-based manga retrieval using manga109 dataset [J].
Matsui, Yusuke ;
Ito, Kota ;
Aramaki, Yuji ;
Fujimoto, Azuma ;
Ogawa, Toru ;
Yamasaki, Toshihiko ;
Aizawa, Kiyoharu .
MULTIMEDIA TOOLS AND APPLICATIONS, 2017, 76 (20) :21811-21838
[10]   Deep Learning Algorithm for Satellite Imaging Based Cyclone Detection [J].
Shakya, Snehlata ;
Kumar, Sanjeev ;
Goswami, Mayank .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2020, 13 :827-839