Overview of the Ninth Dialog System Technology Challenge: DSTC9

被引：0

作者：

Gunasekara, Chulaka ^{[1
]}

Kim, Seokhwan ^{[2
]}

D'Haro, Luis Fernando ^{[3
]}

Rastogi, Abhinav ^{[4
]}

Chen, Yun-Nung ^{[5
]}

Eric, Mihail ^{[2
]}

Hedayatnia, Behnam ^{[2
]}

Gopalakrishnan, Karthik ^{[2
]}

Liu, Yang ^{[2
]}

Huang, Chao-Wei ^{[5
]}

Hakkani-Tur, Dilek ^{[6
]}

Li, Jinchao ^{[7
]}

Zhu, Qi ^{[8
]}

Luo, Lingxiao ^{[8
]}

Liden, Lars ^{[9
]}

Huang, Kaili ^{[10
]}

Shayandeh, Shahin ^{[11
]}

Liang, Runze ^{[8
]}

Peng, Baolin ^{[12
]}

Zhang, Zheng ^{[8
]}

Shukla, Swadheen ^{[9
,15
]}

Huang, Minlie ^{[16
]}

Gao, Jianfeng ^{[13
]}

Mehri, Shikib ^{[14
]}

Feng, Yulan ^{[15
]}

Gordon, Carla ^{[16
]}

Alavi, Seyed Hossein ^{[17
]}

Traum, David

Eskenazi, Maxine ^{[18
]}

Beirami, Ahmad ^{[4
]}

Cho, Eunjoon ^{[19
]}

Crook, Paul A. ^{[20
]}

De, Ankita ^{[21
]}

Geramifard, Alborz ^{[21
]}

Kottur, Satwik ^{[20
]}

Moon, Seungwhan ^{[22
]}

Poddar, Shivani ^{[4
]}

Subba, Rajen ^{[23
]}

机构：

[1] IBM Res Yorktown Hts, Yorktown Hts, NY 10598 USA

[2] Amazon Alexa AI, Sunnyvale, CA 94089 USA

[3] Univ Politecn Madrid, Speech Technol & Machine Learning Grp, ETSI Telecomunicac, Elect Engineer Dept, Madrid 28040, Spain

[4] Google, Google DeepMind, Mountain View 94041, CA USA

[5] Natl Taiwan Univ, Taipei 10617, Taiwan

[6] Univ Illinois, Urbana, IL 61801 USA

[7] Snap Inc, 110th Ave NE, Bellevue 98004, WA USA

[8] Tsinghua Univ, Beijing 100084, Peoples R China

[9] Microsoft Res, One Microsoft Way, Redmond 98054, WA USA

[10] Microsoft, One Microsoft Way, Redmond, WA 98052 USA

[11] Apple AIML Redwood City, Redwood City, CA USA

[12] Microsoft Res Lab, Redmond, WA USA

[13] Microsoft Res Lab, Redmond Microsoft Bldg, Redmond 98052, WA USA

[14] Contextual AI San Jose, San Jose, CA USA

[15] Carnegie Mellon Univ, Pittsburgh, PA 15213 USA

[16] USC ICT, 12015 Waterfront Dr, Playa Vista, CA 90035 USA

[17] Univ British Columbia, Vancouver, BC V6T 1Z4, Canada

[18] Carnegie Mellon Univ, Gates Hillman Complex, Pittsburgh, PA USA

[19] Tofu San Mateo Calif, San Mateo, CA 94401 USA

[20] Meta AI, Meta Platforms, Menlo Pk, CA 94025 USA

[21] Meta, 1 Hacker Way, Menlo Pk, CA 94025 USA

[22] Meta Real Labs, Seattle, WA 98052 USA

[23] Google YouTube, San Carlos, CA USA

来源：

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING | 2024年 / 32卷

关键词：

Task analysis; Measurement; Oral communication; Accuracy; Proposals; Mathematical models; Speech processing; Artificial intelligence; computational intelligence; chatbots; computational linguistics; human-machine systems; human computer interaction; natural language processing; natural language generation; personal voice assistants;

D O I：

10.1109/TASLP.2024.3426331

中图分类号：

O42 [声学];

学科分类号：

070206 ; 082403 ;

摘要：

This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with Unstructured Knowledge Access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog and 4. Situated interactive multimodal dialog. This paper describes the task definition, provided datasets, baselines, and evaluation setup for each track. We also summarize the results of the submitted systems to highlight the general trends of the state-of-the-art technologies for the tasks.

引用

页码：4066 / 4076

页数：11

共 50 条

[1] Interactive Evaluation of Dialog Track at DSTC9
Mehri, Shikib
Feng, Yulan
Gordon, Carla
Alavi, Seyed Hossein
Traum, David
Eskenazi, Maxine
LREC 2022: THIRTEEN INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2022, : 5731 - 5738
[2] Overview of the seventh Dialog System Technology Challenge: DSTC7
Fernando D'Haro, Luis
Yoshino, Koichiro
Hori, Chiori
Marks, Tim K.
Polymenakos, Lazaros
Kummerfeld, Jonathan K.
Galley, Michel
Gao, Xiang
COMPUTER SPEECH AND LANGUAGE, 2020, 62
[3] Overview of the sixth dialog system technology challenge: DSTC6
Horia, Chiori
Perez, Julien
Higashinakac, Ryuichiro
Horia, Takaaki
Boureau, Y-Lan
Inabae, Michimasa
Tsunomorif, Yuiko
Takahashig, Tetsuro
Yoshinoh, Koichiro
Kim, Seokhwan
COMPUTER SPEECH AND LANGUAGE, 2019, 55 : 1 - 25
[4] Overview of the Eighth Dialog System Technology Challenge: DSTC8
Kim, Seokhwan
Galley, Michel
Gunasekara, Chulaka
Lee, Sungjin
Atkinson, Adam
Peng, Baolin
Schulz, Hannes
Gao, Jianfeng
Li, Jinchao
Adada, Mahmoud
Huang, Minlie
Lastras, Luis
Kummerfeld, Jonathan K.
Lasecki, Walter S.
Hori, Chiori
Cherian, Anoop
Marks, Tim K.
Rastogi, Abhinav
Zang, Xiaoxue
Sunkara, Srinivas
Gupta, Raghav
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2021, 29 (29) : 2529 - 2540
[5] Overview of the Tenth Dialog System Technology Challenge: DSTC10
Yoshino, Koichiro
Chen, Yun-Nung
Crook, Paul
Kottur, Satwik
Li, Jinchao
Hedayatnia, Behnam
Moon, Seungwhan
Fei, Zhengcong
Li, Zekang
Zhang, Jinchao
Feng, Yang
Zhou, Jie
Kim, Seokhwan
Liu, Yang
Jin, Di
Papangelis, Alexandros
Gopalakrishnan, Karthik
Hakkani-Tur, Dilek
Damavandi, Babak
Geramifard, Alborz
Hori, Chiori
Shah, Ankit
Zhang, Chen
Li, Haizhou
Sedoc, Joao
D'Haro, Luis F.
Banchs, Rafael
Rudnicky, Alexander
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2024, 32 : 765 - 778
[6] Speech Aware Dialog System Technology Challenge (DSTC11)
Soltau, Hagen
Shafran, Izhak
Wang, Mingqiu
Rastogi, Abhinav
Zhao, Jeffrey
Jia, Ye
Han, Wei
Cao, Yuan
Miranda, Aramys
INTERSPEECH 2023, 2023, : 4668 - 4672
[7] Task-Oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10
Thulke, David
Daheim, Nico
Dugast, Christian
Ney, Hermann
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2024, 32 : 733 - 741
[8] Special Issue on the Eighth Dialog System Technology Challenge
Kim, Seokhwan
Schulz, Hannes
Gunasekara, Chulaka
Hori, Chiori
Rastogi, Abhinav
D'Haro, Luis
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2021, 29 : 2434 - 2436
[9] An overview of the Slovenian spoken dialog system
Ipšic, Ivo
Pavešic, Nikola
Journal of Computing and Information Technology, 2002, 10 (04) : 295 - 301
[10] THE MSIIP SYSTEM FOR DIALOG STATE TRACKING CHALLENGE 5
Su, Ying
Li, Miao
Wu, Ji
2016 IEEE WORKSHOP ON SPOKEN LANGUAGE TECHNOLOGY (SLT 2016), 2016, : 525 - 530

← 1 2 3 4 5 →