Spatial-Temporal Attention Two-Stream Convolution Neural Network for Smoke Region Detection

Smoke detection is of great significance for fire location and fire behavior analysis in a fire video surveillance system. Smoke image classification methods based on a deep convolution network have achieved high accuracy. However, the combustion of different types of fuel can produce smoke with different colors, such as black smoke, grey smoke, and white smoke. Additionally, the diffusion characteristic of smoke can lead to transparent smoke regions accompanied by colors and textures of background objects. Therefore, compared with smoke image classification, smoke region detection is a challenging task. This paper proposes a two-stream convolutional neural network based on spatio-temporal attention mechanism for smoke region segmentation (STCNNsmoke). The spatial stream extracts spatial features of foreground objects using the semi-supervised ranking model. The temporal stream uses optical flow characteristics to represent the dynamic characteristics of smoke such as diffusion and flutter features. Specifically, the spatio-temporal attention mechanism is presented to fuse the spatial and temporal characteristics of smoke and pay more attention to the moving regions with smoke colors and textures by predicting attention weights of channels. Furthermore, the spatio-temporal attention model improves the channel response of smoke-moving regions for the segmentation of complete smoke regions. The proposed method is evaluated and analyzed from multiple perspectives such as region detection accuracy and anti-interference. The experimental results showed that the proposed method significantly improved the ability of segmenting thin smoke and small smoke.