Detection of breath sounds in speech: A deep learning approach

Breath sound detection from speech recordings has wide-ranging applications, from high-quality audio recordings to medical diagnostics. However, perceptual recognition of breath sounds for annotation is prone to errors, and breath sounds typically occupy only 5% of speech recordings, leading to significant class imbalance. Additionally, the limited availability of annotated data makes the application of deep learning (DL) methods challenging. This paper proposes the use of thermal and normal videos, alongside speech data, to mitigate annotation errors in breath sound detection. To address class imbalance, we leverage self- supervised learning (SSL), employing a jigsaw puzzle solver as a pretext task to augment training data and enhance model performance. The jigsaw puzzle solver helps address the class imbalance by creating a balanced task for pretraining, improving the performance of the downstream task. This work also uses convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM) models to locate breath sounds in speech recordings accurately. The proposed SSL implementation achieves an F1-score of 96% in a speaker- independent configuration. The proposed algorithm has also been tested on publicly available audio recordings from YouTube1and the BiLSTM version is available for testing on Hugging Face.2