Mask-guided cycle-GAN for specular highlight removal

Specular highlight removal is an important yet challenging problem in image enhancement. Recent meth-ods based on deep learning and trained by massive paired or unpaired data have demonstrated promising performance for this task. Methods based on unpaired data have recently gained more attention for eas-ier training data collection. In this paper, we present a Mask-Guided Cycle-GAN for specular highlight removal on unpaired data. Incorporating the idea that specular highlight mainly has characteristics in lightness, we attempt to train a module only on luminance channel before considering all channels, and then adopt the training results to guide the subsequent highlight removal module. We further convert the highlight removal problem to image-to-image translation by using cycle-consistent adversarial net-work (Cycle-GAN). In the proposed network, a non-negative matrix factorization (NMF) based method is incorporated to obtain accurate highlight masks. The proposed method is evaluated using the specu-lar highlight image quadruples (SHIQ) and the LIME datasets, and the advantages are demonstrated via comparative experimental results.(c) 2022 Published by Elsevier B.V.