Surveillance audio-based rainfall observation: An enhanced strategy for extreme rainfall observation

The increased frequency of extreme rainfall events (EREs) causing disastrous effects on society has become an indisputable fact in recent years. The weak performance of the current observation network in terms of accuracy and spatiotemporal resolution makes the development of new rainfall estimation techniques essential to reduce the impact of ERE-driven disasters. The rainfall sounds collected by widespread surveillance cameras provide an opportunity to produce high-resolution, all-weather rainfall data. However, the complex structure of the ground surface and the random background noise make building an effective surveillance audio-based ERE estimation system challenging. In this study, from the viewpoint of surveillance sound space, a 3D printer was used to create a shelter for the surveillance camera to define the underlying surface of falling raindrops artificially. Combining the knowledge of meteorology, micro-physics, and acoustics of rainfall, the shelter structure was designed to standardize the acoustical behavior while enhancing the consistency and specificity of raindrop sound, especially in complex scenarios such as those disturbed by different levels of wind. After that, convolutional neural network-based deep learning algorithms were used to classify ERE levels, and an audio-based ERE classification system was built. The experimental results show that the shelter facilitates audio-based rainfall representation; moreover, with the help of shelter, our proposed system achieved performance with about 93.4% accuracy in complex rainfall scenarios. Our study supports high-resolution rainfall data production on existing surveillance resources, developing a novel and reliable alternative for the perception of ERE and the calibration of observations from current rainfall networks.