CNN-Transformer with Stepped Distillation for Fine-Grained Visual Classification

Capturing the fine-grained details and global information are vital for fine-grained visual classification (FGVC). However, most of the existing methods either use the CNN or rely on the Transformer that can not effectively extract the local and long-range features for FGVC. To address this issue, we propose the CNN-Transformer with stepped distillation (SDCT) for FGVC. In this method, we propose the multi-level and multi-scale feature combiner (MLMSFC) for capturing the rich local features. In the MLMSFC, the image-level features of different scales extracted by the backbone CNN are fused with the part-level features of different scales obtained by the parts selector module and the CNN which shares the weights with the backbone. Meanwhile, the Transformer is utilized to extract the global features from the input image. Moreover, to teach the CNNs to learn more global features of images for inference, we propose the stepped distillation (SD) module. Through the stepped-wise distillation, the shallower stage can learn effective features guided by the deeper stage, resulting in the local and global feature representation improvement and generalization capability enhancement. The experiments conducted on the three popular FGVC datasets demonstrate that the proposed SDCT achieves competitive results compared with the state-of-the-art methods.