Generalized Zero-Shot Learning Based on Manifold Alignment

Generalized zero-shot learning is a method that can classify seen and unseen samples by learning training samples' visual and semantic modal information. Visual modal information is generally extracted by feature extraction networks pre-trained with a large-scale data set, and semantic modal information is typically represented by class attributes. Different categories have shared semantic information, therefore, through learning the mapping between two modal information, the transferable knowledge can be used to classify testing samples. However, most methods align the two modal information of the per-sample rather than considering the alignment of the distribution of multiple instances in the two modalities. We utilize variational autoencoders mapping two modalities' information to a shared latent space, then align the samples' manifold structure of them to promote the accuracy of model classification. We evaluate the proposed method on several benchmark datasets (CUB, SUN, and AWA2), and the significant improvements have proved the method's effectiveness.