Computational Super-Resolution Imaging With a Sparse Rotational Camera Array

被引:5
作者
Zhang, Yifei [1 ]
Li, Tianren [1 ]
Zhang, Yu [1 ]
Chen, Peirong [1 ]
Qu, Yufu [1 ]
Wei, Zhenzhong [1 ]
机构
[1] Beihang Univ, Sch Instrumentat & Optoelect Engn, Beijing 100191, Peoples R China
基金
中国国家自然科学基金;
关键词
Cameras; Imaging; Superresolution; Sensors; Sensor arrays; Image resolution; Layout; Camera array; multi-frame super-resolution; hardware-software co-design; physics-enabled DNN; IMAGES;
D O I
10.1109/TCI.2023.3265919
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
A sparse camera array utilizes several identical cameras for multi-frame super-resolution. However, the influence of camera layout on super-resolution performance remains unclear. In conventional camera arrays, all cameras share a similar observation model, resulting in redundancy during information collection. This study presents a novel super-resolution framework that actively controls cameras in a sparse camera array to rotate to certain degrees. Through a detailed analysis of the forward imaging model, it is shown that the rotation of cameras uses the rotational asymmetry property of pixel layouts and shape. The proposed camera layout method provides more diverse and informative spatial sampling than a conventional camera array with only translations between cameras. A physics-enabled deep neural network was introduced for multi-frame super-resolution. With the explicitly defined gradients and affine models as guidance, it can effectively decode additional information collected by the rotational camera array and overcome the large-scale spatial feature alignment problem. The experimental results on synthetic and real prototype datasets validated the proposed theory and solution. We believe our study can inspire innovations for future task-specific camera array system design.
引用
收藏
页码:425 / 434
页数:10
相关论文
共 35 条
[1]   Spatial aliasing for near-field sensor arrays [J].
Abhayapala, TD ;
Kennedy, RA ;
Williamson, RC .
ELECTRONICS LETTERS, 1999, 35 (10) :764-765
[2]  
[Anonymous], 2007, PROC IEEE INT C COMP, DOI DOI 10.1109/ICCV.2007.4408888
[3]   A General and Adaptive Robust Loss Function [J].
Barron, Jonathan T. .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :4326-4334
[4]   Deep learning: step forward to high-resolution in vivo shortwave infrared imaging [J].
Baulin, Vladimir A. ;
Usson, Yves ;
Le Guevel, Xavier .
JOURNAL OF BIOPHOTONICS, 2021, 14 (07)
[5]  
Bhat G., 2021, P IEEE CVF INT C COM, P2460
[6]   Deep Burst Super-Resolution [J].
Bhat, Goutam ;
Danelljan, Martin ;
Van Gool, Luc ;
Timofte, Radu .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, :9205-9214
[7]  
Bonchev S., 2010, PROC IEEE 13 INT C I, P1
[8]   Super-resolution of images based on local correlations [J].
Candocia, FM ;
Principe, JC .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1999, 10 (02) :372-380
[9]   Cross Parallax Attention Network for Stereo Image Super-Resolution [J].
Chen, Canqiang ;
Qing, Chunmei ;
Xu, Xiangmin ;
Dickinson, Patrick .
IEEE TRANSACTIONS ON MULTIMEDIA, 2022, 24 :202-216
[10]   Image Super-Resolution Using Deep Convolutional Networks [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (02) :295-307