Automating the Categorization of Learning Activities, to Help Improve Learning Design

As part of the large-scale implementation of Learning Design at The Open University, the UK's largest higher education institution, a taxonomy of learning activities informs the development of course modules. The taxonomy is also used to map a module's Learning Design, to categorize its learning activities, after it has been developed. This enables course teams to compare a module's Learning Design with student outcomes, in order to determine which Learning Designs are most effective and in which circumstances. However, the mapping process is labor-intensive and open to inconsistencies, making the outcomes less trustworthy and less useful for learning analytics. In this paper, we present an exploratory study that investigates the automatization of the mapping process by means of both unsupervised and supervised machine learning approaches. For the supervised machine learning (Logistic Regression), we use a labelled set of 35,000 activity descriptions classified as either reflective or non-reflective (i.e., whether or not the activity involves student reflection) drawn from 267 modules. Our outcomes, with similar to 79% accuracy, are sufficiently promising for this approach to merit further work, extending it in particular to a larger set of Learning Design activities.