MEKF: long-tailed visual recognition via multiple experts with knowledge fusion

The distribution of data in real-world applications often shows a long-tailed shape, making long-tailed recognition challenging as traditional models bias toward majority categories. Multi-expert ensemble methods have shown promise but often suffer from insufficient expert diversity and high model variance. To address these issues, we propose multiple experts with knowledge fusion (MEKF), which includes diversified fusion experts and dual-view self-distillation. MEKF enhances expert diversity by fusing features of different depths and introduces distribution diversity loss with distribution weights. Dual-view self-distillation reduces model variance by extracting semantic information from weakly augmented data predictions. Experiments on CIFAR100-LT, ImageNet-LT, and Places-LT benchmarks validate MEKF's effectiveness, showing excellent performance.