Semantic Fusion and Contrastive Generation for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) aims to leverage a classifier trained on seen classes to categorize instances from both seen and unseen classes. Several approaches have been introduced to synthesize visual features that simulate those of unseen classes for training classifiers. However, existing methods only emphasize the distributional relationships between synthesized and real features, while neglecting the inter-class relationships among the synthesized features. Consequently, synthesized visual features exhibit significant loose intra-class distributions and numerous outliers. Furthermore, the generator trained solely on seen classes tend to overfit these classes. In this paper, a Semantic Fusion and Contrastive Generation (SFCG) framework is proposed for GZSL. Specifically, a visual-semantic contrastive generation method and a visual features similarity loss are explored to address the challenges of loose intra-class distribution and outliers in synthesized visual features. Moreover, semantic attributes are fused to create novel and diverse semantic instances for training a balanced generator. The SFCG model is evaluated on four widely-used ZSL benchmark datasets: CUB, FLO, AWA2, and SUN. It achieves harmonic mean accuracies of 68.4% on CUB, 71.3% on FLO, 73.4% on AWA2, and 45.2% on SUN, demonstrating the efficacy of the proposed method.