Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario

被引:94
作者
Medennikov, Ivan [1 ,2 ]
Korenevsky, Maxim [1 ]
Prisyach, Tatiana [1 ]
Khokhlov, Yuri [1 ]
Korenevskaya, Mariya [1 ]
Sorokin, Ivan [1 ]
Timofeeva, Tatiana [1 ]
Mitrofanov, Anton [1 ]
Andrusenko, Andrei [2 ]
Podluzhny, Ivan [1 ]
Laptev, Aleksandr [2 ]
Romanenko, Aleksei [1 ,2 ]
机构
[1] STC Innovat Ltd, St Petersburg, Russia
[2] ITMO Univ, St Petersburg, Russia
来源
INTERSPEECH 2020 | 2020年
关键词
speaker diarization; TS-VAD; CHiME-6;
D O I
10.21437/Interspeech.2020-1602
中图分类号
R36 [病理学]; R76 [耳鼻咽喉科学];
学科分类号
100104 ; 100213 ;
摘要
Speaker diarization for real-life scenarios is an extremely challenging problem. Widely used clustering-based diarization approaches perform rather poorly in such conditions, mainly due to the limited ability to handle overlapping speech. We propose a novel Target-Speaker Voice Activity Detection (TS-VAD) approach, which directly predicts an activity of each speaker on each time frame. TS-VAD model takes conventional speech features (e.g., MFCC) along with i-vectors for each speaker as inputs. A set of binary classification output layers produces activities of each speaker. I-vectors can be estimated iteratively, starting with a strong clustering-based diarization. We also extend the TS-VAD approach to the multi-microphone case using a simple attention mechanism on top of hidden representations extracted from the single-channel TS-VAD model. Moreover, post-processing strategies for the predicted speaker activity probabilities are investigated. Experiments on the CHiME-6 unsegmented data show that TS-VAD achieves state-of-the-art results outperforming the baseline x-vector-based system by more than 30% Diarization Error Rate (DER) abs.
引用
收藏
页码:274 / 278
页数:5
相关论文
共 34 条
[1]   The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines [J].
Barker, Jon ;
Watanabe, Shinji ;
Vincent, Emmanuel ;
Trmal, Jan .
19TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2018), VOLS 1-6: SPEECH RESEARCH FOR EMERGING MARKETS IN MULTILINGUAL SOCIETIES, 2018, :1561-1565
[2]   Front-End Factor Analysis for Speaker Verification [J].
Dehak, Najim ;
Kenny, Patrick J. ;
Dehak, Reda ;
Dumouchel, Pierre ;
Ouellet, Pierre .
IEEE TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2011, 19 (04) :788-798
[3]  
Delcroix M, 2018, 2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), P5554, DOI 10.1109/ICASSP.2018.8462661
[4]   BUT system for DIHARD Speech Diarization Challenge 2018 [J].
Diez, Mireia ;
Landini, Federico ;
Burget, Lukas ;
Rohdin, Johan ;
Silnova, Anna ;
Zmolikova, Katerina ;
Novotny, Ondrej ;
Vesely, Karel ;
Glembek, Ondrej ;
Plchot, Oldrich ;
Mosner, Ladislav ;
Matejka, Pavel .
19TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2018), VOLS 1-6: SPEECH RESEARCH FOR EMERGING MARKETS IN MULTILINGUAL SOCIETIES, 2018, :2798-2802
[5]   Developing On-Line Speaker Diarization System [J].
Dimitriadis, Dimitrios ;
Fousek, Petr .
18TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2017), VOLS 1-6: SITUATED INTERACTION, 2017, :2739-2743
[6]  
Ding SJ, 2020, Arxiv, DOI arXiv:1908.04284
[7]  
Drude L., 2018, 13 ITG FACHT SPRACHK
[8]  
Fredouille C, 2009, PROC RT 2009 NIST RI
[9]  
Fujita Y, 2019, 2019 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU 2019), P296, DOI [10.1109/ASRU46091.2019.9003959, 10.1109/asru46091.2019.9003959]
[10]  
Garcia-Romero D, 2017, INT CONF ACOUST SPEE, P4930, DOI 10.1109/ICASSP.2017.7953094