Effects of Text Difficulty and Readers on Predicting Reading Comprehension from Eye Movements

The task of predicting reader state from readers' eye gaze is not trivial. Whilst eye movements have long been shown to reflect the reading process, the task of predicting quantified measures of reading comprehension has been attempted with unsatisfactory results. We conducted an experiment to collect eye gaze data from participants as they read texts with differing degrees of difficulty. Participants were sourced as being either first or second English language readers. We investigated the effects that reader background and text difficulty have predicting reading comprehension. The results indicate that prediction rates are similar for first and second language readers. The best combination is where the concept level is one level higher than the readability level. The optimal predictors are ELM+NN and Random Forests as they consistently produced the lowest MSEs on average. These findings are a promising step forward to predicting reading comprehension. The intention is to use such predictions in adaptive eLearning environments.