Video super-resolution using controlled subpixel detector shifts

被引:96
作者
Ben-Ezra, M
Zomet, A
Nayar, SK
机构
[1] Siemens Corp Res, Princeton, NJ 08540 USA
[2] Columbia Univ, Dept Comp Sci, New York, NY 10027 USA
关键词
sensors; jitter camera; jitter video; super-resolution; motion blur;
D O I
10.1109/TPAMI.2005.129
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Video cameras must produce images at a reasonable frame-rate and with a reasonable depth of field. These requirements impose fundamental physical limits on the spatial resolution of the image detector. As a result, current cameras produce videos with a very low resolution. The resolution of videos can be computationally enhanced by moving the camera and applying super-resolution reconstruction algorithms. However, a moving camera introduces motion blur, which limits super-resolution quality. We analyze this effect and derive a theoretical result showing that motion blur has a substantial degrading effect on the performance of super-resolution. The conclusion is that, in order to achieve the highest resolution, motion blur should be avoided. Motion blur can be minimized by sampling the space-time volume of the video in a specific manner. We have developed a novel camera, called the "jitter camera," that achieves this sampling. By applying an adaptive super-resolution algorithm to the video produced by the jitter camera, we show that resolution can be notably enhanced for stationary or slowly moving objects, while it is improved slightly or left unchanged for objects with fast and complex motions. The end result is a video that has a significantly higher resolution than the captured one.
引用
收藏
页码:977 / 987
页数:11
相关论文
共 20 条
[1]  
[Anonymous], P EUR C COMP VIS ECC
[2]   Limits on super-resolution and how to break them [J].
Baker, S ;
Kanade, T .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2002, 24 (09) :1167-1183
[3]  
BASCLE B, 1996, P 4 EUR C COMP VIS, V2, P573
[4]  
CAPEL D, 2000, P INT C PATT REC, V1, P600
[5]   Efficient super-resolution via image warping [J].
Chiang, MC ;
Boult, TE .
IMAGE AND VISION COMPUTING, 2000, 18 (10) :761-771
[6]   Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images [J].
Elad, M ;
Feuer, A .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 1997, 6 (12) :1646-1658
[7]   IMPROVING RESOLUTION BY IMAGE REGISTRATION [J].
IRANI, M ;
PELEG, S .
CVGIP-GRAPHICAL MODELS AND IMAGE PROCESSING, 1991, 53 (03) :231-239
[8]   Fundamental limits of reconstruction-based superresolution algorithms under local translation [J].
Lin, ZC ;
Shum, HY .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2004, 26 (01) :83-97
[9]   ROBUST REGRESSION METHODS FOR COMPUTER VISION - A REVIEW [J].
MEER, P ;
MINTZ, D ;
ROSENFELD, A ;
KIM, DY .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 1991, 6 (01) :59-70
[10]  
*MIN, 2005, DIM A1