Understanding the Influence of Intelligent Agents on Students' Sense of Classroom Belonging, Engagement, and Academic Performance in Online Classes

As universities continue to expand their offering of online courses, it is imperative for researchers to identify pedagogy that supports student learning. At the same time, the application of artificial intelligence as a tool to support student learning is growing. For example, one recent trend in learning management systems is the integration of intelligent agents, which allow instructors to program automatic, targeted e-mails to students based on particular release conditions (i.e., a targeted congratulations e-mail when a student earns an A). Despite the growing popularity of intelligent agents and their potential to support students in asynchronous online courses, researchers have not yet investigated their influence on students. Using a quasi-experimental design, this study sought to examine the impact of intelligent agents on students' academic engagement, sense of belonging, and academic performance. Two large (i.e., 150 students) asynchronous online courses were randomly assigned to either the intervention (intelligent agent messages; n = 113) or control (whole class announcements; n = 122). To compare the two groups, we administered a survey composed of Likert-scale and short-answer questions measuring academic engagement and sense of belonging at the beginning and end of the semester. Analyses revealed no significant differences between the groups' engagement, belonging, or academic performance; however, all students reported a significant decrease in behavioral and emotional engagement across the semester. Qualitative analyses suggested that the intervention and control groups did seem to describe the sense of belonging differently, which may indicate that the use of intelligent agents influenced students' perceptions of belonging.