Effect of background Indian music on performance of speech recognition models for Hindi databases

Multimedia content analysis has shown great interest over the past few decades. One of the works which find great attention to the researchers is automatic speech recognition (ASR) of speech data from broadcast radio and TV program. However, the presence of background music in such data heavily degrades the performance of ASR models. In this paper, we initially studied the temporal and spectral properties of music samples recorded from five different Indian instruments. Further, to see the effect of background Indian music on the recognition efficiency of ASR models for Hindi databases, these speech models were trained on both isolated and continuous speech databases using both clean and noisy databases. Hence, a total of four scenarios were considered: 1. Clean Isolated Database, 2. Noisy Isolated Database, 3. Clean Continuous Database, 4. Noisy Continuous Database. The variation of ASR performance was observed for different SNR levels of background music (0–30 dB). These background noises were combined with clean speech signals both independently where the sound of a single instrument was used as well as in combination with each other where sounds from several instruments were mixed. Overall, maximum degradation in performance of ASR is observed for background noise generated from audio samples of Been with an average WER of 13.37 and 72.21 for isolated and continuous text models whereas minimum degradation in performance of ASR is observed for background noise generated from audio samples of Harmonium and Flute with a WER of 15.25 and 66.09 for isolated text models and continuous text models respectively. We further correlated the observed results of ASR performance to the temporal and spectral properties of the music signals and found that higher values of Zero Crossing Rate, Roll-off rate, spectral centroid and spectral flux indicated greater degradation in ASR performance. Hence, these features are found to give important cues to understand the background noise as compared to other features like spectral entropy and Short Term Energy. The work presented in this paper will be useful in better understanding of music compensation algorithms focused on the Indian market.