SmokeSeger: A Transformer-CNN coupled model for urban scene smoke segmentation

Smoke is an informative indicator of early fire and gas leakage. Segmenting the smoke from images can provide detailed information about the smoke volume, dispersion direction, and source location, which has significant implications considering the proliferation of video surveillance systems in cities. Focusing on smoke segmentation in the urban scene, we designed a dual-branch segmentation model, named SmokeSeger, which couples a Transformer branch and a CNN branch to enhance the representation of both global and local features. To address the lack of real-scene smoke datasets, we built an urban scene smoke segmentation dataset containing 3217 images of fire smoke and exhaust emissions with accurate annotations. Experiments validate that the SmokeSeger outperforms other mainstream segmentation methods on the proposed dataset. Visualization of attention maps reveals that the model could effectively capture the semantic relationship between the smoke and the corresponding source, which benefits the discrimination between smoke and smoke-like objects.