A Novel Loss Function Based on Clustering Quality Criteria in Spatio-Temporal Clustering

Video instance segmentation has diverse applications in the autonomous vehicle industry, image surveillance systems, production lines, and medical video analysis. There are two approaches, top-down and bottom-up, to address the task of instance segmentation in video. Top-down methods heavily rely on image-level segmentation and have separate processes for detection and tracking. They show a strong dependency on image-level segmentation. Bottom-up approaches leverage both spatial and temporal information simultaneously, aiming for a gradual transition from pixel-level features to spatio-temporal instances. This paper introduces a novel method to improve the performance of video instance segmentation based on the bottom-up approach. In this method, by utilizing the silhouette metric to assess clustering quality and introducing the central distance metric in loss functions, the values of embedding vectors are improved, leading to the generation of more distinct clusters in space and time. Experimental results demonstrate that this method achieved an approximately 2% improvement compared to the baseline method.