EvLSD-IED: Event-Based Line Segment Detection With Image-to-Event Distillation

Event cameras, also known as neuromorphic cameras, have garnered significant interest in recent years because of their high temporal resolution for capturing dynamic scenes. Their unique asynchronous triggering mechanism based on illumination changes offers a unique advantage in capturing edge information, particularly in line segment detection tasks. However, directly applying state-of-the-art image-based line segment detection methods to events poses a nontrivial challenge due to the lack of essential semantic information (such as color and texture), the inherent scarcity of information in areas with low light variation, as well as the absence of well-annotated datasets. To address these issues, we introduce EvLSD-IED, a learning-based method for event-based line segment detection, along with an image-to-event distillation (IED) framework for feature boosting, to compensate for the limitations due to the characteristics above of event cameras. Specifically, we propose a scene-level distillation method to help the event-based modal build a high-level understanding of the scene. In addition, to improve the model's sensitivity on low-contrast edges, we propose a line-level distillation method, enabling the model to generate comprehensive features even with incomplete input information. In addition, we build two datasets, i.e., a synthetic dataset named E-wireframe and the first real-world dataset called real-scene line segment detection (RE-LSD), serving for training and evaluation. Experimental evaluations on the two collected datasets validate the effectiveness of EvLSD-IED, showcasing its ability to achieve accurate and robust line segment detection even under extreme conditions. With pretraining on the E-wireframe and fine-tuning on RE-LSD, the model achieves a mean structural average precision (msAP) of 47.8%, significantly surpassing the performance of the state of the art. Our code, pretrained models, and datasets are available at https://github.com/Qiuben/EvLSD-IED.