Full-Spectrum Out-of-Distribution Detection

Existing out-of-distribution (OOD) detection literature clearly defines semantic shift as a sign of OOD but does not have a consensus over covariate shift. Samples experiencing covariate shift but not semantic shift from the in-distribution (ID) are either excluded from the test set or treated as OOD, which contradicts the primary goal in machine learning-being able to generalize beyond the training distribution. In this paper, we take into account both shift types and introduce full-spectrum OOD (F-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and design three benchmarks. These new benchmarks have a more fine-grained categorization of distributions (i.elet@tokeneonedot, training ID, covariate-shifted ID, near-OOD, and far-OOD) for the purpose of more comprehensively evaluating the pros and cons of algorithms. To address the F-OOD detection problem, we propose SEM, a simple featurebased semantics score function. SEM is mainly composed of two probability measures: one is based on high-level features containing both semantic and non-semantic information, while the other is based on low-level feature statistics only capturing non-semantic image styles. With a simple combination, the non-semantic part is canceled out, which leaves only semantic information in SEM that can better handle F-OOD detection. Extensive experiments on the three new benchmarks show that SEM significantly outperforms current state-of-the-art methods. Our code and benchmarks are released in https://github.com/ Jingkang50/OpenOOD.