Relighting Images in the Wild with a Self-Supervised Siamese Auto-Encoder

被引:8
|
作者
Liu, Yang [1 ]
Neophytou, Alexandros [2 ]
Sengupta, Sunando [2 ]
Sommerlade, Eric [2 ]
机构
[1] Univ Surrey, Guildford, Surrey, England
[2] Microsoft Corp, Reading, Berks, England
来源
2021 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2021) | 2021年
关键词
D O I
10.1109/WACV48630.2021.00008
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose a self-supervised method for image relighting of single view images in the wild. The method is based on an auto-encoder which deconstructs an image into two separate encodings, relating to the scene illumination and content, respectively. In order to disentangle this embedding information without supervision, we exploit the assumption that some augmentation operations do not affect the image content and only affect the direction of the light. A novel loss function, called spherical harmonic loss, is introduced that forces the illumination embedding to convert to a spherical harmonic vector. We train our model on large-scale datasets such as Youtube 8M and CelebA. Our experiments show that our method can correctly estimate scene illumination and realistically re-light input images, without any supervision or a prior shape model. Compared to supervised methods, our approach has similar performance and avoids common lighting artifacts.
引用
收藏
页码:32 / 40
页数:9
相关论文
共 50 条
  • [41] Semi-Supervised Adversarial Auto-Encoder to Expedite Human Activity Recognition
    Thapa, Keshav
    Seo, Yousung
    Yang, Sung-Hyun
    Kim, Kyong
    SENSORS, 2023, 23 (02)
  • [42] Regularized Masked Auto-Encoder for Semi-Supervised Hyperspectral Image Classification
    Wang, Liguo
    Wang, Heng
    Wang, Peng
    Wang, Lifeng
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62
  • [43] A semi-supervised auto-encoder using label and sparse regularizations for classification
    Chai, Zhilei
    Song, Wei
    Wang, Huiling
    Liu, Fei
    APPLIED SOFT COMPUTING, 2019, 77 : 205 - 217
  • [44] An Efficient Encryption and Compression of Sensed IoTMedical Images Using Auto-Encoder
    El-kafrawy, Passent
    Aboghazalah, Maie
    Ahmed, Abdelmoty M.
    Torkey, Hanaa
    El-Sayed, Ayman
    CMES-COMPUTER MODELING IN ENGINEERING & SCIENCES, 2023, 134 (02): : 909 - 926
  • [45] Self-Supervised Variational Auto-Encoders
    Gatopoulos, Ioannis
    Tomczak, Jakub M.
    ENTROPY, 2021, 23 (06)
  • [46] Ensemble Stacked Auto-encoder Classification on LIDAR Remote Sensing Images
    Li, Dawei
    Zhang, Ruifang
    JOURNAL OF THE INDIAN SOCIETY OF REMOTE SENSING, 2018, 46 (04) : 597 - 604
  • [47] Ensemble Stacked Auto-encoder Classification on LIDAR Remote Sensing Images
    Dawei Li
    Ruifang Zhang
    Journal of the Indian Society of Remote Sensing, 2018, 46 : 597 - 604
  • [48] Near-Infrared Spectral Modeling Based on Stacked Supervised Auto-Encoder
    Sun Zhi-xing
    Zhao Zhong-gai
    Liu Fei
    SPECTROSCOPY AND SPECTRAL ANALYSIS, 2022, 42 (03) : 749 - 756
  • [49] A semi-supervised deep auto-encoder based intrusion detection for iot
    Fenanir S.
    Semchedine F.
    Harous S.
    Baadache A.
    Fenanir, Samir (samir.fenanir@univ-setif.dz), 2020, International Information and Engineering Technology Association (25): : 569 - 577
  • [50] Pivotal Auto-Encoder via Self-Normalizing ReLU
    Goldenstein, Nelson
    Sulam, Jeremias
    Romano, Yaniv
    IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2024, 72 : 3201 - 3212