An Efficient Multiscale Spatial Rearrangement MLP Architecture for Image Restoration

The effective use of long-range information canyield improved network performance, which is very importantfor image restoration. Although local window-based modelshave linear complexity and can be feasibly applied to processhigh-resolution images, a single-scale window has a limitedreceptive field and is less efficient for encoding long-range contextinformation. To address this issue, this paper presents a single-stage multiscale spatial rearrangement multilayer perceptron(MSSR-MLP) architecture that can obtain information at dif-ferent scales within a local window. Specifically, we proposea simple and efficient spatial rearrangement module (SRM)that moves information outside the local window to the insideof the local window so that long-range dependencies can bemodeled using only a window-based fully connected (FC) layer.The SRM can extend the local receptive field of a window-based FC layer without introducing additional parameters andFLOPs. Utilizing several spatial rearrangement modules withdifferent step sizes, we design an efficient multiscale spatial rearrangement MLP architecture for image restoration. Thisdesign aggregates multiscale information to achieve improvedrestoration quality while maintaining a low computational cost.Extensive experiments conducted on several image restorationtasks demonstrate the efficiency and effectiveness of our method.For example, it requires only similar to 4.3% of the FLOPs neededby SwinIR for Gaussian gray image denoising,similar to 13.9% ofthe FLOPs needed by C(2)PNet for single-image dehazing and similar to 18.9% of the FLOPs needed by MAXIM for single-imagemotion deblurring but achieves better performance on each ofthese restoration tasks