Machine-Learned Computational Models Can Enhance the Study of Text and Discourse: A Case Study Using Eye Tracking to Model Reading Comprehension

We propose that machine-learned computational models (MLCMs), in which the model parameters and perhaps even structure are learned from data, can complement extant approaches to the study of text and discourse. Such models are particularly useful when theoretical understanding is insufficient, when the data are rife with nonlinearities and interactivity, and when researchers aspire to take advantage of "big data." Being fully instantiated computer programs, MLCMs can also be used for autonomous assessment and real-time intervention. We illustrate these ideas in the context of an eye movement-based MLCM of textbase comprehension during reading along connected text. Using a dataset where 104 participants read a 6,500-word text, we trained Random Forests models to predict comprehension scores from six eye movement features. The models were highly accurate (area under the receiver operating characteristic curve = .902; r = .661), robust, and generalized across participants, suggesting possible use in future studies. We conclude by arguing for an increased role of MLCMs in the future of discourse research.