Open Set Recognition in Real World

Open set recognition (OSR) constitutes a critical endeavor within the domain of computer vision, frequently deployed in applications, such as autonomous driving and medical imaging recognition. Existing OSR methodologies predominantly center on the acquisition of a profound association between image data and corresponding labels, facilitating the extraction of discriminative features instrumental for distinguishing novel categories. Nevertheless, real-world scenarios often introduce not only novel classes (referred to semantic shift) but also intricate environmental modifications that engender alterations in the distribution of established classes (termed as covariate shift). The latter phenomenon has the potential to undermine the robust correlation between images and labels established by conventional statistical correlation modeling approaches, consequently resulting in significant degradation of OSR performance. Causal correlation stands as the fundamental linkage between entities, routinely harnessed by humans to enhance their cognitive capacities for a more profound comprehension of the intricate world. With inspiration drawn from this perspective, our work herein introduces the causal inference-inspired open set recognition (CISOR) approach tailored for real-world OSR (RWOSR). CISOR represents the pioneering initiative to leverage the stability inherent in causal correlation to construct two pivotal modules: the covariate causal independence (CCI) module and the semantic causal uniqueness (SCU) module, both instrumental in addressing the RWOSR problem. The CCI module adeptly confronts the challenge of covariate shift by imposing constraints on the correlations between inter-class causal features. This strategy effectively mitigates the impact of spurious correlations between distinct categories on the generalization capacity of discriminative features. Furthermore, in order to counteract the issue of semantic shift, the SCU module harnesses correlations between causal features within the same class as constraints, thereby facilitating the extraction of resilient causal features endowed with superior discriminative capabilities. Empirical findings substantiate the superior efficacy of the proposed CIOSR method when compared to state-of-the-art approaches across diverse RWOSR benchmark datasets. The source code of this article will be available at https://github.com/yangzhen1252/RWOSR1.