Accent Recognition Using a Spectrogram Image Feature-Based Convolutional Neural Network

被引:6
作者
Cetin, Onursal [1 ]
机构
[1] Bandirma Onyedi Eylul Univ, Elect & Elect Engn Dept, TR-10200 Balikesir, Turkey
关键词
Regional accent recognition; Spectrogram; Convolutional neural network; Transfer learning; I-vector; SOUND EVENT CLASSIFICATION; FREQUENCY-CHARACTERISTICS; CNN;
D O I
10.1007/s13369-022-07086-9
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Accent recognition is a significant area of research, whose importance has increased in recent years. Numerous studies have been carried out using various languages to improve the performance of accent recognition systems. However, the recognition of a language's regional accents is still a challenging problem. In this study, regional accents of British English were recognized for both gender-independent and gender-dependent experiments using a convolutional neural network. Many different acoustic features were used in the studies. While there is still no generally accepted feature set, the selection of handcrafted features is a challenging task. Moreover, converting audio signals into images in the most appropriate way is critical for a convolutional neural network, a deep learning model commonly used in image applications. To take advantage of the convolutional neural networks' ability to characterize two-dimensional signals, spectrogram image features that visualize the speech signal frequency distribution were used. For this purpose, sound signals were first segmented to their state before normalization. Each segment was combined by taking the fast Fourier transform. The absolute value was taken, and then, the log function was used to compress the dynamic range of these linear rate maps, resulting in log-power rate maps. After a grayscale image was formed by normalizing the obtained time-frequency matrix in the range of [0, 1], the dynamic range was quantified to red, green, and blue color values to generate a monochrome image. Thus, the feature extraction process, which is time-consuming and challenging, was simplified using spectrogram images and a convolutional neural network. In addition, although it is desired that the training and test data have a uniform distribution, the heterogeneity of the data adversely affects the performance of machine learning algorithms. To overcome this problem and improve the model's performance, transfer learning, a state-of-the-art technology that enables data transfer from the pre-trained AlexNet model with 1.3 million pictures on the ImageNet database, was utilized. Several performance metrics, such as accuracy, specificity, sensitivity, precision, and F-score, were used to evaluate the proposed approach. The accuracy of 92.92 and 93.38% and the F-score of 92.67 and 93.19% were obtained for gender-independent and gender-dependent experiments, respectively. Additionally, i-vector-based linear discriminant analysis and support vector machine methods were used in the study. Thus, the results obtained to evaluate the performance of the proposed recognition method are presented comparatively.
引用
收藏
页码:1973 / 1990
页数:18
相关论文
共 95 条
  • [91] A theory of transfer learning with applications to active learning
    Yang, Liu
    Hanneke, Steve
    Carbonell, Jaime
    [J]. MACHINE LEARNING, 2013, 90 (02) : 161 - 189
  • [92] Simultaneous fault diagnosis of wind turbine using multichannel convolutional neural networks
    Zare, Samira
    Ayati, Moosa
    [J]. ISA TRANSACTIONS, 2021, 108 : 230 - 239
  • [93] Accent Recognition with Hybrid Phonetic Features
    Zhang, Zhan
    Wang, Yuehai
    Yang, Jianyi
    [J]. SENSORS, 2021, 21 (18)
  • [94] Speech emotion recognition using deep 1D & 2D CNN LSTM networks
    Zhao, Jianfeng
    Mao, Xia
    Chen, Lijiang
    [J]. BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2019, 47 : 312 - 323
  • [95] A Comprehensive Survey on Transfer Learning
    Zhuang, Fuzhen
    Qi, Zhiyuan
    Duan, Keyu
    Xi, Dongbo
    Zhu, Yongchun
    Zhu, Hengshu
    Xiong, Hui
    He, Qing
    [J]. PROCEEDINGS OF THE IEEE, 2021, 109 (01) : 43 - 76