How well do collaboration quality estimation models generalize across authentic school contexts?

被引:4
作者
Chejara, Pankaj [1 ]
Kasepalu, Reet [1 ]
Prieto, Luis P. [1 ]
Rodriguez-Triana, Maria Jesus [1 ]
Ruiz Calleja, Adolfo [1 ]
Schneider, Bertrand [1 ]
机构
[1] Tallinn Univ, Tallinn, Estonia
关键词
collaboration quality; computer-supported collaborative learning; generalizability; machine learning; multimodal learning analytics; KNOWLEDGE; SIGNALS; ONLINE;
D O I
10.1111/bjet.13402
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Multimodal learning analytics (MMLA) research has made significant progress in modelling collaboration quality for the purpose of understanding collaboration behaviour and building automated collaboration estimation models. Deploying these automated models in authentic classroom scenarios, however, remains a challenge. This paper presents findings from an evaluation of collaboration quality estimation models. We collected audio, video and log data from two different Estonian schools. These data were used in different combinations to build collaboration estimation models and then assessed across different subjects, different types of activities (collaborative-writing, group-discussion) and different schools. Our results suggest that the automated collaboration model can generalize to the context of different schools but with a 25% degradation in balanced accuracy (from 82% to 57%). Moreover, the results also indicate that multimodality brings more performance improvement in the case of group-discussion-based activities than collaborative-writing-based activities. Further, our results suggest that the video data could be an alternative for understanding collaboration in authentic settings where higher-quality audio data cannot be collected due to contextual factors. The findings have implications for building automated collaboration estimation systems to assist teachers with monitoring their collaborative classrooms.What is already known about this topicWhat does this paper addImplications for practicePractitioners notesMultimodal learning analytics researchers have established several features as potential indicators for collaboration quality, e.g., speaking time or joint visual attention.The current state of the art has shown the feasibility of building automated collaboration quality models.Recent research has provided preliminary evidence of the generalizability of developed automated models across contexts different in terms of given task and subject.This paper offers collaboration indicators for different types of collaborative learning activities in authentic classroom settings.The paper includes a systematic investigation into collaboration quality automated model's generalizability across different tasks, types of tasks and schools.This paper also offers a comparison between different modalities' potential to estimate collaboration quality in authentic settings.The findings inform the development of automated collaboration monitoring systems for authentic classroom settings.This paper provides evidence on across-school generalizability capabilities of collaboration quality estimation models.
引用
收藏
页码:1602 / 1624
页数:23
相关论文
共 55 条
[31]   MTFeedback: Providing Notifications to Enhance Teacher Awareness of Small Group Work in the Classroom [J].
Martinez-Maldonado, Roberto ;
Clayphan, Andrew ;
Yacef, Kalina ;
Kay, Judy .
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, 2015, 8 (02) :187-200
[32]   Capturing and analyzing verbal and physical collaborative learning interactions at an enriched interactive tabletop [J].
Martinez-Maldonado, Roberto ;
Dimitriadis, Yannis ;
Martinez-Mones, Alejandra ;
Kay, Judy ;
Yacef, Kalina .
INTERNATIONAL JOURNAL OF COMPUTER-SUPPORTED COLLABORATIVE LEARNING, 2013, 8 (04) :455-485
[33]  
Ochoa X., 2017, HDB LEARNING ANAL
[34]   Temporal analysis of multimodal data to predict collaborative learning outcomes [J].
Olsen, Jennifer K. ;
Sharma, Kshitij ;
Rummel, Nikol ;
Aleven, Vincent .
BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY, 2020, 51 (05) :1527-1547
[35]   Multi-modal Social Signal Analysis for Predicting Agreement in Conversation Settings [J].
Ponce-Lopez, Vctor ;
Escalera, Sergio ;
Baro, Xavier .
ICMI'13: PROCEEDINGS OF THE 2013 ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2013, :495-501
[36]   Literature Review on Co-Located Collaboration Modeling Using Multimodal Learning Analytics-Can We Go the Whole Nine Yards? [J].
Praharaj, Sambit ;
Scheffel, Maren ;
Drachsler, Hendrik ;
Specht, Marcus .
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, 2021, 14 (03) :367-385
[37]  
Pugh S. L., 2022, LAK22 12 INT LEARNIN
[38]  
Reilly J. M., 2019, EDM 2019 P 12 INT C
[39]   Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings [J].
Rummel, N ;
Spada, H .
JOURNAL OF THE LEARNING SCIENCES, 2005, 14 (02) :201-241
[40]  
Rummel N, 2011, COMPUT-SUPP COLLAB L, P367, DOI 10.1007/978-1-4419-7710-6_17