AI-assisted analysis of content, structure, and sentiment in MOOC discussion forums

被引:3
|
作者
Yee, Michael [1 ]
Roy, Anindya [2 ]
Perdue, Meghan [2 ]
Cuevas, Consuelo [1 ]
Quigley, Keegan [1 ]
Bell, Ana [3 ]
Rungta, Ahaan [2 ]
Miyagawa, Shigeru [4 ]
机构
[1] MIT Lincoln Lab, Artificial Intelligence Technol Grp, Lexington, MA 02421 USA
[2] MIT, Open Learning, Cambridge, MA USA
[3] MIT, Dept Elect Engn & Comp Sci, Cambridge, MA USA
[4] MIT, Dept Linguist & Philosophy, Cambridge, MA USA
关键词
MOOCs; discussion forums; forum posts; natural language processing; text classification; machine learning; transformers; artificial intelligence;
D O I
10.3389/feduc.2023.1250846
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Discussion forums are a key component of online learning platforms, allowing learners to ask for help, provide help to others, and connect with others in the learning community. Analyzing patterns of forum usage and their association with course outcomes can provide valuable insight into how learners actually use discussion forums, and suggest strategies for shaping forum dynamics to improve learner experiences and outcomes. However, the fine-grained coding of forum posts required for this kind of analysis is a manually intensive process that can be challenging for large datasets, e.g., those that result from popular MOOCs. To address this issue, we propose an AI-assisted labeling process that uses advanced natural language processing techniques to train machine learning models capable of labeling a large dataset while minimizing human annotation effort. We fine-tune pretrained transformer-based deep learning models on category, structure, and emotion classification tasks. The transformer-based models outperform a more traditional baseline that uses support vector machines and a bag-of-words input representation. The transformer-based models also perform better when we augment the input features for an individual post with additional context from the post's thread (e.g., the thread title). We validate model quality through a combination of internal performance metrics, human auditing, and common-sense checks. For our Python MOOC dataset, we find that annotating approximately 1% of the forum posts achieves performance levels that are reliable for downstream analysis. Using labels from the validated AI models, we investigate the association of learner and course attributes with thread resolution and various forms of forum participation. We find significant differences in how learners of different age groups, gender, and course outcome status ask for help, provide help, and make posts with emotional (positive or negative) sentiment.
引用
收藏
页数:17
相关论文
共 43 条
  • [1] Is critical thinking happening? Testing content analysis schemes applied to MOOC discussion forums
    O'Riordan, Tim
    Millard, David E.
    Schulz, John
    COMPUTER APPLICATIONS IN ENGINEERING EDUCATION, 2021, 29 (04) : 690 - 709
  • [2] Bringing Order to Chaos in MOOC Discussion Forums with Content-Related Thread Identification
    Wise, Alyssa Friend
    Cui, Yi
    Vytasek, Jovita
    LAK '16 CONFERENCE PROCEEDINGS: THE SIXTH INTERNATIONAL LEARNING ANALYTICS & KNOWLEDGE CONFERENCE,, 2016, : 188 - 197
  • [3] Mining opinions on LMOOCs: Sentiment and content analyses of Chinese students' comments in discussion forums
    Peng, Jian-E
    Jiang, Yuanlan
    SYSTEM, 2022, 109
  • [4] Exploring the relationship between social presence and learners ' prestige in MOOC discussion forums using automated content analysis and social network analysis
    Zou, Wenting
    Hu, Xiao
    Pan, Zilong
    Li, Chenglu
    Cai, Ying
    Liu, Min
    COMPUTERS IN HUMAN BEHAVIOR, 2021, 115
  • [5] AI-assisted Cyber Security Exercise Content Generation: Modeling a Cyber Conflict
    Zacharis, Alexandros
    Gavrila, Razvan
    Patsakis, Constantinos
    Ikonomou, Demosthenes
    2023 15TH INTERNATIONAL CONFERENCE ON CYBER CONFLICT, CYCON, 2023, : 217 - 238
  • [6] Setting the pace: examining cognitive processing in MOOC discussion forums with automatic text analysis
    Moore, Robert L.
    Oliver, Kevin M.
    Wang, Chuang
    INTERACTIVE LEARNING ENVIRONMENTS, 2019, 27 (5-6) : 655 - 669
  • [7] A parallel neural network structure for sentiment classification of MOOCs discussion forums
    Gao, Yi
    Sun, Xia
    Wang, Xin
    Guo, Shouxi
    Feng, Jun
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2020, 38 (04) : 4915 - 4927
  • [8] AI-Analyst: An AI-Assisted SDLC Analysis Framework for Business Cost Optimization
    Faruqui, Nuruzzaman
    Thatoi, Priyabrata
    Choudhary, Rohit
    Roncevic, Ivana
    Alqahtani, Hamed
    Sarker, Iqbal H.
    Khanam, Shapla
    IEEE ACCESS, 2024, 12 : 195188 - 195203
  • [9] Understanding COVID-19 Impacts on the Health Workforce: AI-Assisted Open-Source Media Content Analysis
    Pienkowska, Anita
    Ravaut, Mathieu
    Mammadova, Maleyka
    Ang, Chin-Siang
    Wang, Hanyu
    Ong, Qi Chwen
    Bojic, Iva
    Qin, Vicky Mengqi
    Sumsuzzman, Dewan Md
    Ajuebor, Onyema
    Boniol, Mathieu
    Bustamante, Juana Paola
    Campbell, James
    Cometto, Giorgio
    Fitzpatrick, Siobhan
    Kane, Catherine
    Joty, Shafiq
    Car, Josip
    JMIR FORMATIVE RESEARCH, 2024, 8
  • [10] Intelligent AI Assisted Psychological Disorder Analysis Using Sentiment Inference
    Kamath, Anil
    Raje, Nirav
    Konduri, Saishashank
    Shah, Hardik
    Naik, Varsha
    Bhattacharjee, Krishnanjan
    Shivakarthik, S.
    Mehta, Swati
    Kumar, Ajai
    2018 INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTING, COMMUNICATIONS AND INFORMATICS (ICACCI), 2018, : 24 - 29