Can artificial intelligence (AI) help in computer science education? A meta-analysis approach

Several studies have investigated the effect of Artificial Intelligence (AI) on students' learning achievement in education. However, limited research targeted Computer Science (CS) education, which is considered crucial regardless of the future profession. Consequently, scant information exists on how AI might impact students' learning achievement in CS education. To address this research gap, this study conducts a systematic review and a meta-analysis to investigate how AI integration affects learning achievement in CS education and the potential moderating variables of this effect. Specifically, 28 studies (n = 2765 participants in total) were included and meta-analyzed, and the obtained effect size was very large (g = 1.36, p <.001). Particularly, intelligent tutoring systems (ITSs) were found to have the highest effect (g = 1.45; huge) as an AI technology. Additionally, the AI intervention duration and the geographical distribution of students are found to moderate the AI effect in CS education. The findings of this study can serve as a reference for various stakeholders (e. g., educators, computer scientists, instructional designers) on how to integrate AI and improve learning experiences and outcomes in CS education.