Deep learning for real-time single-pixel video

被引:262
作者
Higham, Catherine F. [1 ]
Murray-Smith, Roderick [1 ]
Padgett, Miles J. [2 ]
Edgar, Matthew P. [2 ]
机构
[1] Univ Glasgow, Sch Comp Sci, Glasgow G12 8QQ, Lanark, Scotland
[2] Univ Glasgow, Sch Phys & Astron, Glasgow G12 8QQ, Lanark, Scotland
基金
英国工程与自然科学研究理事会; 欧洲研究理事会;
关键词
D O I
10.1038/s41598-018-20521-y
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Single-pixel cameras capture images without the requirement for a multi-pixel sensor, enabling the use of state-of-the-art detector technologies and providing a potentially low-cost solution for sensing beyond the visible spectrum. One limitation of single-pixel cameras is the inherent trade-off between image resolution and frame rate, with current compressive (compressed) sensing techniques being unable to support real-time video. In this work we demonstrate the application of deep learning with convolutional auto-encoder networks to recover real-time 128 x 128 pixel video at 30 frames-per-second from a single-pixel camera sampling at a compression ratio of 2%. In addition, by training the network on a large database of images we are able to optimise the first layer of the convolutional network, equivalent to optimising the basis used for scanning the image intensities. This work develops and implements a novel approach to solving the inverse problem for single-pixel cameras efficiently and represents a significant step towards real-time operation of computational imagers. By learning from examples in a particular context, our approach opens up the possibility of high resolution for task-specific adaptation, with importance for applications in gas sensing, 3D imaging and metrology.
引用
收藏
页数:9
相关论文
共 25 条
[1]  
[Anonymous], 2011, INT C ART INT STAT
[2]   Learning Deep Architectures for AI [J].
Bengio, Yoshua .
FOUNDATIONS AND TRENDS IN MACHINE LEARNING, 2009, 2 (01) :1-127
[3]   Large-Scale Machine Learning with Stochastic Gradient Descent [J].
Bottou, Leon .
COMPSTAT'2010: 19TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL STATISTICS, 2010, :177-186
[4]  
Candes E. J., 2006, INT C MATH, V3, P1433, DOI DOI 10.4171/022-3/69
[5]   Near-optimal signal recovery from random projections: Universal encoding strategies? [J].
Candes, Emmanuel J. ;
Tao, Terence .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2006, 52 (12) :5406-5425
[6]   HADAMARD-TRANSFORM IMAGE SCANNING [J].
DECKER, JA .
APPLIED OPTICS, 1970, 9 (06) :1392-+
[7]   Image Super-Resolution Using Deep Convolutional Networks [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (02) :295-307
[8]   Compressed sensing [J].
Donoho, DL .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2006, 52 (04) :1289-1306
[9]   Single-pixel imaging via compressive sampling [J].
Duarte, Marco F. ;
Davenport, Mark A. ;
Takhar, Dharmpal ;
Laska, Jason N. ;
Sun, Ting ;
Kelly, Kevin F. ;
Baraniuk, Richard G. .
IEEE SIGNAL PROCESSING MAGAZINE, 2008, 25 (02) :83-91
[10]   Simultaneous real-time visible and infrared video with single-pixel detectors [J].
Edgar, Matthew. P. ;
Gibson, Graham M. ;
Bowman, Richard W. ;
Sun, Baoqing ;
Radwell, Neal ;
Mitchell, Kevin J. ;
Welsh, Stephen S. ;
Padgett, Miles J. .
SCIENTIFIC REPORTS, 2015, 5