Ray tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video

被引:0
|
作者
Kim, Youngwook [1 ]
Yun, Jehyeong [2 ]
Yun, Joungil [3 ]
Kwak, Sangwoon [3 ]
Ihm, Insung [1 ]
机构
[1] Sogang Univ, Dept Comp Sci & Engn, Seoul, South Korea
[2] MIDAS Informat Technol, CIM Dev Team 1, Seongnam, South Korea
[3] Elect & Telecommun Res Inst, Media Res Div, Daejeon, South Korea
基金
新加坡国家研究基金会;
关键词
Immersive video; Real-time stereoscopic rendering; Disocclusion; 3D background model; Ray tracing; Virtual reality; VIEW-SYNTHESIS; COMPRESSION;
D O I
10.1007/s10055-023-00921-w
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Immersive video stored in multiview video-plus-depth format can provide viewers with vivid immersive experiences. However, rendering such video in real time in immersive environments remains a challenging task due to the high resolution and refresh rate demanded by recent extended reality displays. An essential issue in this immersive rendering is the disocclusion problem that inevitably occurs when virtual views are synthesized via the de facto standard 3D warping technique. In this paper, we present a novel virtual view synthesis framework that, from a live immersive video stream, renders stereoscopic images in real time for a freely moving virtual viewer. The main difference from previous approaches is that the surrounding background environment of the immersive video's virtual scene is progressively reproduced on the fly directly in the 3D space while the input stream is being rendered. To allow this, we propose a new 3D background modeling scheme that, based on GPU-accelerated real-time ray tracing, efficiently and incrementally builds the background model in compact 3D triangular mesh. Then, we demonstrate that the 3D background environment can effectively alleviate the critical disocclusion problem in the immersive rendering, eventually reducing spatial and temporal aliasing artifacts. It is also suggested that the 3D representation of background environment enables extension of the virtual environment of immersive video by interactively adding 3D visual effects during rendering.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] Ray tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video
    Youngwook Kim
    Jehyeong Yun
    Joungil Yun
    Sangwoon Kwak
    Insung Ihm
    Virtual Reality, 2024, 28
  • [2] Real-time video-based modeling and rendering of 3D scenes
    Naemura, T
    Tago, J
    Harashima, H
    IEEE COMPUTER GRAPHICS AND APPLICATIONS, 2002, 22 (02) : 66 - 73
  • [3] Real-Time Stereoscopic Rendering of Realistic Avatar for Interactive 3D Telepresence System
    Wang, Yanbin
    Khoo, Eng Tat
    2017 2ND INTERNATIONAL CONFERENCE ON IMAGE, VISION AND COMPUTING (ICIVC 2017), 2017, : 577 - 581
  • [4] A study on the possibility of implementing a real-time stereoscopic 3D rendering TV system
    Kim, Hyeongseok
    Lee, Jaesung
    DISPLAYS, 2015, 40 : 24 - 34
  • [5] The Real-Time Stereoscopic 3D Image Conversion of Photo and Video Contents
    Lee, Jun-Eui
    Lee, Sang-Won
    Lee, Won-Hyung
    FUTURE INFORMATION TECHNOLOGY, PT II, 2011, 185 : 337 - 344
  • [6] Real-time 3D video compression for tele-immersive environments
    Yang, ZY
    Cui, Y
    Anwar, Z
    Bocchino, R
    Kiyanclar, N
    Nahrstedt, K
    Campbell, RH
    Yurcik, W
    MULTIMEDIA COMPUTING AND NETWORKING 2006, 2006, 6071
  • [7] Real-time 3D rendering with hatching
    Jordane Suarez
    Farès Belhadj
    Vincent Boyer
    The Visual Computer, 2017, 33 : 1319 - 1334
  • [8] Real-time 3D rendering with hatching
    Suarez, Jordane
    Belhadj, Fares
    Boyer, Vincent
    VISUAL COMPUTER, 2017, 33 (10): : 1319 - 1334
  • [9] Real-Time 3D Rendering Proces sor for 2D-to-3D Conversion of Stereoscopic Displays
    Lai, Yeong-Kang
    Chung, Yu-Chieh
    2014 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS (ICCE), 2014, : 119 - 120
  • [10] A study on the real-time toon rendering for 3D geometry model
    Kang, D
    Kim, D
    Yoon, K
    FIFTH INTERNATIONAL CONFERENCE ON INFORMATION VISUALISATION, PROCEEDINGS, 2001, : 391 - 396