A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration

Image restoration under adverse weather conditions is of great significancc for the subsequent advanced Computer vision tasks. However, most existing image restoration algorithms only removc singlc weather degradation, and few studies has been conducted on all-in-one weather-degraded image restoration. The denoising diffusion probability modcl is combined with Vision Transformer to proposc a Transformer-based diffusion model for all-in-one weather-degraded image restoration. First, the weather-degraded image is utilized as the condition to guide the reverse sampling of diffusion model and generate corresponding clean background image. Then, the subspace transposed Transformer for noise estimation (NE-STT) is proposed, which utilizes the dcgraded image and the noisy State to estimate noise distribution, including the subspace transposed self-attention (STSA) mechanism and a dual grouped gated feed-forward network (DGGFFN). The STSA adopts subspace transformation coefficient to effectively capture global long-range dependencies while significantly reducing computational bürden. The DGGFFN employs the dual grouped gated mechanism to enhance the nonlinear characterization ability of feed-forward network. The experimental results show that in comparison with the recently developed algorithms, such as All-in-One and TransWeather, the method proposed obtains a Performance gain of 3. 68 and 3. 08 dB in average peak signal-to-noise ratio while 2. 93% and 3. 13% in average structural similarity on 5 weather-dcgraded datasets. © 2024 Shanghai Jiaotong University. All rights reserved.