Challenge variance: Exploiting format differences for personalized learner models

In this study, we present an approach to utilizing variance in students' performance across different formats (multiple-choice, numeric input, word problems) as a target for personalization. We have developed a measure called challenge variance, that indicates the degree to which different formats pose varying levels of challenge for individual learners. We investigated whether challenge variance could be a useful source of information for developing learner models by analyzing data from an online math tutoring platform. Results demonstrated that challenge variance has a relationship with an external activity, indicating its utility as a means of predicting how well a learner will perform in a new setting. We discuss the affordances and issues with the measure and whether or not it could be a useful additional tool in developing personalized learner models as an intuitive and platform-agnostic measure of performance.