Accurately Deciphering Novel Cell Type in Spatially Resolved Single-Cell Data Through Optimal Transport

Recent advances in spatial transcriptomics enable the detection of spatial heterogeneity at single-cell resolution. However, existing annotation methods are limited in performance due to that they are mainly designed for scRNA-seq data without accounting for spatial coordinate information. More importantly, they have been struggling to identify novel cell types. Here, we introduce SPOTAnno, a novel method that allows for the simultaneous and accurate identification of both seen and novel cell types within spatially resolved single-cell data using Optimal Transport (OT). Concretely, SPOTAnno first embeds the spatial data into low-dimensional embeddings through the transformer accounting for spatial coordinates. Based on the low-dimensional embeddings, SPOTAnno employs a partial alignment strategy to remove batch effects by aligning target data to the reference prototypes through OT-based statistical information. In parallel, SPOTAnno utilizes an OT-based representation learning mechanism to map each cell onto the prototypes of the target data, which enhances global cluster discrimination and ensures local cell consistency within the target dataset. Additionally, an entropy-based loss is applied for target cells to increase the prediction certainty. Comprehensive experiments demonstrate that SPOTAnno outperforms state-of-the-art methods in both intra-data and cross-data settings, showcasing its effectiveness in cell type discovery and annotation accuracy. Implementations are available at https://github.com/QingJun3/SPOTAnno.