Focal Aggregation Transformer for Light Field Image Super-Resolution

Transformer has achieved significant progress in light field image super-resolution (LFSR) due to its long-range dependency learning ability for inter-intra view feature aggregation. However, locality information of each sub-aperture view is ignored in intra-view and inter-view aggregation with Transformer, hampering the high-quality light field image reconstruction. To this end, we propose a global to local aggregation approach termed Focal Aggregation for LFSR. In particular, Focal Aggregation includes two strategies: inter-view global to local aggregation (InterG2L) and intra-view global to local aggregation (IntraG2L). InterG2L is proposed to obtain complementary information from different views. IntraG2L is developed to extract efficient representations of a single sub-aperture view. InterG2L and IntraG2L are organized in a cascade way so that the global information of the input can be gathered for each sub-aperture image in a coarse to fine aggregation approach. Meanwhile, we also develop a global to local hierarchical feature aggregation approach named HierG2L, which enhances the last hierarchical feature used for light field reconstruction according to the input. Based on the above three global to local aggregation strategies, we construct a focal aggregation transformer (FAT) for LFSR. Experiments are performed on commonly-used LFSR benchmarks. Results demonstrate that FAT achieves superior results compared with other leading methods on synthesized and real data.