LIGHT-FIELD VIEW SYNTHESIS USING A CONVOLUTIONAL BLOCK ATTENTION MODULE

被引:12
作者
Gul, M. Shahzeb Khan [1 ]
Mukati, M. Umair [2 ]
Baetz, Michel [1 ]
Forchhammer, Soren [2 ]
Keinert, Joachim [1 ]
机构
[1] Fraunhofer IIS, Moving Picture Technol, D-91058 Erlangen, Germany
[2] Tech Univ Denmark, DTU Foton, DK-2800 Lyngby, Denmark
来源
2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP) | 2021年
关键词
Light-field; View synthesis; Deep-learning;
D O I
10.1109/ICIP42928.2021.9506586
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Consumer light-field (LF) cameras suffer from a low or limited resolution because of the angular-spatial trade-off. To alleviate this drawback, we propose a novel learning-based approach utilizing attention mechanism to synthesize novel views of a light-field image using a sparse set of input views (i.e., 4 corner views) from a camera array. In the proposed method, we divide the process into three stages, stereo-feature extraction, disparity estimation, and final image refinement. We use three sequential convolutional neural networks for each stage. A residual convolutional block attention module (CBAM) is employed for final adaptive image refinement. Attention modules are helpful in learning and focusing more on the important features of the image and are thus sequentially applied in the channel and spatial dimensions. Experimental results show the robustness of the proposed method. Our proposed network outperforms the state-of-the-art learning-based light-field view synthesis methods on two challenging real-world datasets by 0.5 dB on average. Furthermore, we provide an ablation study to substantiate our findings.
引用
收藏
页码:3398 / 3402
页数:5
相关论文
共 30 条
  • [1] [Anonymous], 2020, P IEEE INT C MULT EX
  • [2] [Anonymous], 2018, RAYTRIX LIGHT FILED
  • [3] DeepView: View synthesis with learned gradient descent
    Flynn, John
    Broxton, Michael
    Debevec, Paul
    DuVall, Matthew
    Fyffe, Graham
    Overbeck, Ryan
    Snavely, Noah
    Tucker, Richard
    [J]. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 2362 - 2371
  • [4] Gul M.S.K., 2019, 30 BRIT MACHINE VISI, P23
  • [5] Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks
    Gul, M. Shahzeb Khan
    Gunturk, Bahadir K.
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (05) : 2146 - 2159
  • [6] Hu J, 2018, PROC CVPR IEEE, P7132, DOI [10.1109/TPAMI.2019.2913372, 10.1109/CVPR.2018.00745]
  • [7] Jin J, 2020, AAAI CONF ARTIF INTE, V34, P11141
  • [8] Learning-Based View Synthesis for Light Field Cameras
    Kalantari, Nima Khademi
    Wang, Ting-Chun
    Ramamoorthi, Ravi
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2016, 35 (06):
  • [9] Toward Bridging the Simulated-to-Real Gap: Benchmarking Super-Resolution on Real Data
    Kohler, Thomas
    Batz, Michel
    Naderi, Farzad
    Kaup, Andre
    Maier, Andreas
    Riess, Christian
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020, 42 (11) : 2944 - 2959
  • [10] Linear View Synthesis Using a Dimensionality Gap Light Field Prior
    Levin, Anat
    Durand, Fredo
    [J]. 2010 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2010, : 1831 - 1838