Semantic-diversity transfer network for generalized zero-shot learning via inner disagreement based OOD detector

Zero-shot learning (ZSL) aims to recognize objects from unseen classes, where the key is to transfer knowledge from seen classes to unseen classes by establishing appropriate mappings between visual and semantic features. Currently, the knowledge transfer in many existing works is rather limited due to various factors: (i) the widely used visual features are global ones, and they are not completely consistent with semantic attributes; (ii) only one mapping is learned, which is not able to effectively model diverse visual-semantic relations; (iii) the bias problem in the generalized ZSL (GZSL) could not be effectively handled. In this paper, we propose two techniques to alleviate these limitations. Firstly, we propose a Semantic-diversity transfer Network (SetNet) addressing the first two limitations, where (1) a multiple-attention architecture and a diversity regularizer are proposed to learn multiple local visual features being more consistent with semantic attributes and (2) a projector ensemble which geometrically takes diverse local features as inputs is proposed to diversify visual-semantic relations. Secondly, we propose an inner disagreement based domain detection module (ID3M) for GZSL to alleviate the bias problem, which picks out unseen-class data before class-level classification. Due to the lack of unseen-class data in the training stage, ID3M employs a novel self-contained training scheme and detects out unseen-class data based on a proposed inner disagreement criterion. Experimental results on three public datasets show that the proposed SetNet with the explored ID3M achieves a significant improvement against many state-of-the-art methods. (C) 2021 Elsevier B.V. All rights reserved.