Neural Network Based Light Field Predictors: An Evaluation

Processing and storing the 4D structure of light fields can be challenging and expensive due to the high-dimensional data and its unique characteristics. There are plenty of works employing convolutional neural networks (CNNs) for light field prediction and encoding. Nonetheless, to the best of our knowledge, the literature lacks an efficiency evaluation of different CNN architectures as well as 4D neural networks for these purposes. Therefore, this paper presents an experimental study that assesses the performance of pipeline and U-net convolutional neural networks for light field block prediction in both spatial and angular dimensions. Additionally, we compare these architectures with a novel 4D network that aims to exploit the light field data structure. The results of the study show that U-net and 4D networks outperform classical CNN architectures in terms of prediction accuracy and residue generation. Furthermore, the spatial dimension prediction provides more valuable information for the networks to learn, improving their prediction by 5dB.