Application and Improvement of Dual-Branch Feature Fusion Network in Multi-Class Classification of Gastrointestinal Images

The multi-classification task of gastrointestinal images faces the challenges of recognizing pathological features of different sizes and extracting complex features. To address these problems, this paper proposes a two-branch feature fusion network that combines ResNet50 with Feature Pyramid Network (FPN) and Vision Mamba model. After the two branches extract features separately, they are fused by a designed feature fusion module that includes spatial attention, channel attention, and residual connectivity, which is ultimately used for the classification task. FPN captures global and local features through a high-resolution feature pyramid, and Vision Mamba enhances the recognition of pathological features of different sizes through its unique design. The two-branch feature fusion network leverages the strengths of ResNet50, FPN, and Vision Mamba to effectively deal with the complexity of gastrointestinal images. Experiments on Kvasir-Dataset and Gastrovision datasets show that the improved model proposed in this paper outperforms the existing good models in performance, proving the effectiveness and practicality of the method.