A survey on image and video stitching

被引：9
作者：
LYU W. [1 ]
ZHOU Z. [1 ]
CHEN L. [1 ]
ZHOU Y. [1 ]
机构：
[1] State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing
来源：
Virtual Reality and Intelligent Hardware | 2019年 / 1卷 / 01期
基金：
中国国家自然科学基金;
关键词：
3D stitching; Alignment; Deep learning; Image stitching; Mesh optimization; Panoramic stitching; Registration; Video stitching;
D O I：
10.3724/SP.J.2096-5796.2018.0008
中图分类号：
学科分类号：
摘要：
Image/video stitching is a technology for solving the field of view (FOV) limitation of images/ videos. It stitches multiple overlapping images/videos to generate a wide-FOV image/video, and has been used in various fields such as sports broadcasting, video surveillance, street view, and entertainment. This survey reviews image/video stitching algorithms, with a particular focus on those developed in recent years. Image stitching first calculates the corresponding relationships between multiple overlapping images, deforms and aligns the matched images, and then blends the aligned images to generate a wide-FOV image. A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions. Video stitching is the further extension of image stitching. It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms, and the subsequent frames can then be stitched according to the template. Video stitching is more complicated with moving objects or violent camera movement, because these factors introduce jitter, shakiness, ghosting, and blurring. Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring, while video stabilization algorithms are adopted to solve the jitter and shakiness. This paper further discusses panoramic stitching as a special-extension of image / video stitching. Panoramic stitching is currently the most widely used application in stitching. This survey reviews the latest image/video stitching methods, and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms. Image/video stitching faces long-term challenges such as wide baseline, large parallax, and low-texture problem in the overlapping region. New technologies may present new opportunities to address these issues, such as deep learning-based semantic correspondence, and 3D image stitching. Finally, this survey discusses the challenges of image/video stitching and proposes potential solutions. © 2019 Beijing Zhongke Journal Publishing Co. Ltd.
引用
收藏
页码：55 / 83
页数：28
相关论文
共 72 条
  • [1] Szeliski R., Image Alignment and Stitching: A Tutorial, Foundations and Trends® in Computer Graphics and Vision, 2, 1, pp. 1-104, (2007)
  • [2] Kaynig V., Fischer B., Buhmann J.M., Probabilistic image registration and anomaly detection by nonlinear warping, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, (2008)
  • [3] Silva R.M.A., Gomes P.B., Frensh T., Monteiro D., Real time 360° video stitching and streaming, ACM SIGGRAPH 2016 Posters, pp. 1-2, (2016)
  • [4] Peleg S., Rousso B., Rav-Acha A., Zomet A., Mosaicing on adaptive manifolds, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 10, pp. 1144-1154, (2000)
  • [5] Levin A., Zomet A., Peleg S., Weiss Y., Seamless image stitching in the gradient domain, Computer Vision-ECCV 2004: 2004// 2004, pp. 377-389, (2004)
  • [6] Zomet A., Levin A., Peleg S., Weiss Y., Seamless image stitching by minimizing false edges, IEEE Transactions on Image Processing, 15, 4, pp. 969-977, (2006)
  • [7] Jia J., Tang C.K., Eliminating structure and intensity misalignment in image stitching, Tenth IEEE International Conference on Computer Vision (ICCV'05), pp. 1651-1658, (2005)
  • [8] Brown M., Lowe D.G., Recognizing Panoramas, Proceedings of the IEEE International Conference on Computer Vision, (2003)
  • [9] Brown M., Lowe D.G., Automatic panoramic image stitching using invariant features, International Journal of Computer Vision, 74, 1, pp. 59-73, (2007)
  • [10] Gao J., Kim S.J., Brown M.S., Constructing image panoramas using dual-homography warping, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 49-56, (2011)
12345678