Thank you for attention: A survey on attention-based artificial neural networks for automatic speech recognition

被引：0
作者：
Karmakar, Priyabrata [1 ]
Teng, Shyh Wei [1 ]
Lu, Guojun [2 ]
机构：
[1] Federat Univ, Inst Innovat Sci & Sustainabil, Ballarat, Australia
[2] Federat Univ, Global Profess Sch, Ballarat, Australia
来源：
INTELLIGENT SYSTEMS WITH APPLICATIONS | 2024年 / 23卷
关键词：
Automatic speech recognition (ASR); Attention mechanism; Recurrent neural network (RNN); Transformer; Offline ASR; Streaming ASR; SELF-ATTENTION;
D O I：
10.1016/j.iswa.2024.200406
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on how attention models have grown and changed for offline and streaming speech recognition in recurrent neural networks and Transformer-based systems.
引用
收藏
页数：12
相关论文
共 95 条
[1]  
[Anonymous], 2017, P 8 INT JOINT C NATU
[2]  
Ba J.L., 2016, arXiv preprint arXiv:1607.06450, DOI DOI 10.48550/ARXIV.1607.06450
[3]  
Bahdanau D, 2016, Arxiv, DOI arXiv:1409.0473
[4]  
Bandanau D, 2016, INT CONF ACOUST SPEE, P4945, DOI 10.1109/ICASSP.2016.7472618
[5]  
Bengio Y., 2014, P NIPS WORKSH DEEP L
[6]  
Bird S, 2006, P COLING ACL 2006 IN, DOI [10.48550/arXiv.cs/0205028, DOI 10.3115/1225403.1225421, 10.3115/1225403.1225421]
[7]   "Masks do not work": COVID-19 misperceptions and theory-driven corrective strategies on Facebook [J].
Borah, Porismita ;
Kim, Sojung ;
Hsu, Ying-Chia .
ONLINE INFORMATION REVIEW, 2023, 47 (05) :880-905
[8]   On Online Attention-based Speech Recognition and Joint Mandarin Character-Pinyin Training [J].
Chan, William ;
Lane, Ian .
17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, :3404-3408
[9]  
Chan W, 2016, INT CONF ACOUST SPEE, P4960, DOI 10.1109/ICASSP.2016.7472621
[10]  
Chaudhari S, 2021, Arxiv, DOI [arXiv:1904.02874, DOI 10.48550/ARXIV.1904.02874]
12345678910