Identifying factors influencing online learning outcomes for middle-school students - a re-examination based on XGBoost and SHAP

Accurately identifying the factors that influence the online learning of middle-school students is crucial for improving its effectiveness. However, existing studies rarely focus on the factors influencing the learning outcomes of middle-school students, and traditional methods often struggle to identify the nonlinear relationships among those factors. This study aims to fill this gap, exploring the factors influencing online learning outcomes for middle-school students and their nonlinear impacts. Therefore, the study constructed a CIPP (Context, Input, Process, and Product) theoretical framework, and integrated the XGBoost (Extreme Gradient Boosting) and SHAP (SHapley Additive exPlanations) methods to reveal the nonlinear impact of various factors on online learning outcomes and explore the interactive effects between these factors, based on the questionnaire responses of 142,514 middle-school students. The results indicate the following: (1) Affective engagement is a key factor influencing online learning outcomes. (2) The top four factors - affective engagement, social engagement, self-regulation strategy, and teaching presence - show a nonlinear trend from stable to improved online learning outcomes. Meanwhile, information literacy, the fifth key factor, displays a U-shaped nonlinear trend. (3) Both interactions involving affective engagement - one with social engagement and the other with self-regulation strategy - are beneficial for middle-school students' online learning. This study holds significant value as it has the potential to enhance educational practices, thereby improving student learning experiences and outcomes.