The Challenge of Noisy Classrooms: Speaker Detection During Elementary Students' Collaborative Dialogue

被引:8
作者
Ma, Yingbo [1 ]
Wiggins, Joseph B. [1 ]
Celepkolu, Mehmet [1 ]
Boyer, Kristy Elizabeth [1 ]
Lynch, Collin [2 ]
Wiebe, Eric [2 ]
机构
[1] Univ Florida, Gainesville, FL 32601 USA
[2] North Carolina State Univ, Raleigh, NC 27606 USA
来源
ARTIFICIAL INTELLIGENCE IN EDUCATION (AIED 2021), PT I | 2021年 / 12748卷
基金
美国国家科学基金会;
关键词
Adaptive and intelligent collaborative learning support; Classroom environment; Speaker detection; Multimodal learning; SUPPORT;
D O I
10.1007/978-3-030-78292-4_22
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Adaptive and intelligent collaborative learning support systems are effective for supporting learning and building strong collaborative skills. This potential has not yet been realized within noisy classroom environments, where automated speech recognition (ASR) is very difficult. A key challenge is to differentiate each learner's speech from the background noise, which includes the teachers' speech as well as other groups' speech. In this paper, we explore a multimodal method to identify speakers by using visual and acoustic features from ten video recordings of children pairs collaborating in an elementary school classroom. The results indicate that the visual modality was better for identifying the speaker when in-group speech was detected, while the acoustic modality was better for differentiating in-group speech from background speech. Our analysis also revealed that recurrent neural network (RNN)-based models outperformed convolutional neural network (CNN)-based models with higher speaker detection F-1 scores. This work represents a critical step toward the classroom deployment of intelligent systems that support collaborative learning.
引用
收藏
页码:268 / 281
页数:14
相关论文
共 47 条
[1]   Investigating Help-Giving Behavior in a Cross-Platform Learning Environment [J].
Ahmed, Ishrat ;
Mawasi, Areej ;
Wang, Shang ;
Wylie, Ruth ;
Bergner, Yoav ;
Whitehurst, Amanda ;
Walker, Erin .
ARTIFICIAL INTELLIGENCE IN EDUCATION (AIED 2019), PT I, 2019, 11625 :14-25
[2]   OpenFace 2.0: Facial Behavior Analysis Toolkit [J].
Baltrusaitis, Tadas ;
Zadeh, Amir ;
Lim, Yao Chong ;
Morency, Louis-Philippe .
PROCEEDINGS 2018 13TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE & GESTURE RECOGNITION (FG 2018), 2018, :59-66
[3]   A Study of Automatic Speech Recognition in Noisy Classroom Environments for Automated Dialog Analysis [J].
Blanchard, Nathaniel ;
Brady, Michael ;
Olney, Andrew M. ;
Glaus, Marci ;
Sun, Xiaoyi ;
Nystrand, Martin ;
Samei, Borhan ;
Kelly, Sean ;
D'Mello, Sidney .
ARTIFICIAL INTELLIGENCE IN EDUCATION, AIED 2015, 2015, 9112 :23-33
[4]   Domain-Independent Extraction of Scientific Concepts from Research Articles [J].
Brack, Arthur ;
D'Souza, Jennifer ;
Hoppe, Anett ;
Auer, Soeren ;
Ewerth, Ralph .
ADVANCES IN INFORMATION RETRIEVAL, ECIR 2020, PT I, 2020, 12035 :251-266
[5]  
Celepkolu M., 2021, International Journal of Child-Computer Interaction, V27, DOI [10.1016/j.ijcci.2020.100232, DOI 10.1016/J.IJCCI.2020.100232]
[6]  
Devlin J, 2019, Arxiv, DOI [arXiv:1810.04805, DOI 10.48550/ARXIV.1810.04805]
[7]  
ELAN, ARCHIVE MPI NL TLA E
[8]   Examinations of identity invariance in facial expression adaptation [J].
Ellamil, Melissa ;
Susskind, Joshua M. ;
Anderson, Adam K. .
COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE, 2008, 8 (03) :273-281
[9]   Slow is Good: The Effect of Diligence on Student Performance in the Case of an Adaptive Learning System for Health Literacy [J].
Fadljevic, Leon ;
Maitz, Katharina ;
Kowald, Dominik ;
Pammer-Schindler, Viktoria ;
Gasteiger-Klicpera, Barbara .
LAK20: THE TENTH INTERNATIONAL CONFERENCE ON LEARNING ANALYTICS & KNOWLEDGE, 2020, :112-117
[10]  
FFmpeg, FFMPEG FFMPEG