AI-assisted analysis of content, structure, and sentiment in MOOC discussion forums

被引:3
作者
Yee, Michael [1 ]
Roy, Anindya [2 ]
Perdue, Meghan [2 ]
Cuevas, Consuelo [1 ]
Quigley, Keegan [1 ]
Bell, Ana [3 ]
Rungta, Ahaan [2 ]
Miyagawa, Shigeru [4 ]
机构
[1] MIT Lincoln Lab, Artificial Intelligence Technol Grp, Lexington, MA 02421 USA
[2] MIT, Open Learning, Cambridge, MA USA
[3] MIT, Dept Elect Engn & Comp Sci, Cambridge, MA USA
[4] MIT, Dept Linguist & Philosophy, Cambridge, MA USA
关键词
MOOCs; discussion forums; forum posts; natural language processing; text classification; machine learning; transformers; artificial intelligence;
D O I
10.3389/feduc.2023.1250846
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Discussion forums are a key component of online learning platforms, allowing learners to ask for help, provide help to others, and connect with others in the learning community. Analyzing patterns of forum usage and their association with course outcomes can provide valuable insight into how learners actually use discussion forums, and suggest strategies for shaping forum dynamics to improve learner experiences and outcomes. However, the fine-grained coding of forum posts required for this kind of analysis is a manually intensive process that can be challenging for large datasets, e.g., those that result from popular MOOCs. To address this issue, we propose an AI-assisted labeling process that uses advanced natural language processing techniques to train machine learning models capable of labeling a large dataset while minimizing human annotation effort. We fine-tune pretrained transformer-based deep learning models on category, structure, and emotion classification tasks. The transformer-based models outperform a more traditional baseline that uses support vector machines and a bag-of-words input representation. The transformer-based models also perform better when we augment the input features for an individual post with additional context from the post's thread (e.g., the thread title). We validate model quality through a combination of internal performance metrics, human auditing, and common-sense checks. For our Python MOOC dataset, we find that annotating approximately 1% of the forum posts achieves performance levels that are reliable for downstream analysis. Using labels from the validated AI models, we investigate the association of learner and course attributes with thread resolution and various forms of forum participation. We find significant differences in how learners of different age groups, gender, and course outcome status ask for help, provide help, and make posts with emotional (positive or negative) sentiment.
引用
收藏
页数:17
相关论文
共 43 条
  • [31] A Systematic Review of Advances in AI-Assisted Analysis of Fundus Fluorescein Angiography (FFA) Images: From Detection to Report Generation
    Yu, Tao
    Shao, An
    Wu, Hongkang
    Su, Zichang
    Shen, Wenyue
    Zhou, Jingxin
    Lin, Xingxi
    Shi, Danli
    Grzybowski, Andrzej
    Wu, Jian
    Jin, Kai
    OPHTHALMOLOGY AND THERAPY, 2025, : 599 - 619
  • [32] AI-assisted Real-Time Spatial Delphi: integrating artificial intelligence models for advancing future scenarios analysis
    Yuri Calleo
    Amos Taylor
    Francesco Pilla
    Simone Di Zio
    Quality & Quantity, 2025, 59 (Suppl 2) : 1427 - 1459
  • [33] Impact of AI-assisted CXR analysis in detecting incidental lung nodules and lung cancers in non-respiratory outpatient clinics
    Kwak, Se Hyun
    Kim, Kyeong Yeon
    Choi, Ji Soo
    Kim, Min Chul
    Seol, Chang Hwan
    Kim, Sung Ryeol
    Lee, Eun Hye
    FRONTIERS IN MEDICINE, 2024, 11
  • [34] AI-assisted digital video analysis reveals changes in gait among three-day event horses during competition
    Bucci, Madelyn P.
    Dewberry, L. Savannah
    Staiger, Elizabeth A.
    Allen, Kyle
    Brooks, Samantha A.
    JOURNAL OF EQUINE VETERINARY SCIENCE, 2025, 146
  • [35] Effects of ai-assisted colonoscopy on adenoma miss rate/adenoma detection rate: A protocol for systematic review and meta-analysis
    Shao, Lei
    Yan, Xinzong
    Liu, Chengjiang
    Guo, Can
    Cai, Baojia
    MEDICINE, 2022, 101 (46) : E31945
  • [36] 'AI-navigating' or 'AI-sinking'? An analysis of verbs in research articles titles suspicious of containing AI-generated/assisted content
    Comas-Forgas, Ruben
    Koulouris, Alexandros
    Kouis, Dimitris
    LEARNED PUBLISHING, 2025, 38 (01)
  • [37] Exploring the landscape of AI-assisted decision-making in head and neck cancer treatment: a comparative analysis of NCCN guidelines and ChatGPT responses
    Marchi, Filippo
    Bellini, Elisa
    Iandelli, Andrea
    Sampieri, Claudio
    Peretti, Giorgio
    EUROPEAN ARCHIVES OF OTO-RHINO-LARYNGOLOGY, 2024, 281 (04) : 2123 - 2136
  • [38] Influence of exposure of customized dental implant abutments to different cleaning procedures: an in vitro study using AI-assisted SEM/EDS analysis
    Paul Hofmann
    Andreas Kunz
    Franziska Schmidt
    Florian Beuer
    Dirk Duddeck
    International Journal of Implant Dentistry, 9
  • [39] Can AI-assisted objective facial attractiveness scoring systems replace manual aesthetic evaluations? A comparative analysis of human and machine ratings
    Wang, Ben
    Deng, Yiwen
    Jiang, Haiyue
    JOURNAL OF PLASTIC RECONSTRUCTIVE AND AESTHETIC SURGERY, 2025, 104 : 18 - 25
  • [40] Influence of exposure of customized dental implant abutments to different cleaning procedures: an in vitro study using AI-assisted SEM/EDS analysis
    Hofmann, Paul
    Kunz, Andreas
    Schmidt, Franziska
    Beuer, Florian
    Duddeck, Dirk
    INTERNATIONAL JOURNAL OF IMPLANT DENTISTRY, 2023, 9 (01)