S2Net: Shadow Mask-Based Semantic-Aware Network for Single-Image Shadow Removal

被引:10
作者
Bao, Qiqi [1 ]
Liu, Yunmeng [2 ]
Gang, Bowen [3 ]
Yang, Wenming [1 ]
Liao, Qingmin [1 ]
机构
[1] Tsinghua Univ, Shenzhen Int Grad Sch, Dept Elect Engn, Shenzhen 518055, Peoples R China
[2] Chinese Acad Sci, Shanghai Inst Tech Phys, Key Lab Infrared Syst Detect & Imaging Technol, Shanghai 200031, Peoples R China
[3] Fudan Univ, Sch Management, Dept Stat & Data Sci, Shanghai 200433, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Convolution; Semantics; Task analysis; Kernel; Training; Image color analysis; Shadow removal; semantic-guided feature extraction; semantic transformation; refinement; boundary loss;
D O I
10.1109/TCE.2022.3188968
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Existing shadow removal methods often struggle with two problems: color inconsistencies in shadow areas and artifacts along shadow boundaries. To address these two problems, we propose a novel shadow mask-based semantic-aware network (S(2)Net) that uses shadow masks as guidance for shadow removal. The color inconsistency problem is solved in two steps. First, we use a series of semantic-guided dilated residual (SDR) blocks to transfer statistical information from non-shadow areas to shadow areas. The shadow mask-based semantic transformation (SST) operation in SDR enables the network to remove shadows while keeping non-shadow areas intact. Then, we design a refinement block by incorporating semantic knowledge of shadow masks and applying the learned modulated convolution kernels to get traceless and consistent output. To remove artifacts along shadow boundaries, we propose a newly designed boundary loss. The boundary loss encourages spatial coherence around shadow boundaries. By including the boundary loss as part of the loss function, a significant portion of artifacts along shadow boundaries can be removed. Extensive experiments on the ISTD, ISTD+, SRD and SBU datasets show our S(2)Net outperforms existing shadow removal methods.
引用
收藏
页码:209 / 220
页数:12
相关论文
empty
未找到相关数据