Analyzing learners' engagement and behavior in MOOCs on programming with the Codeboard IDE

Massive Open Online Courses (MOOCs) can be enhanced with the so-called learning-by-doing, designing the courses in a way that the learners are involved in a more active way in the learning process. Within the options for increasing learners' interaction in MOOCs, it is possible to integrate (third-party) external tools as part of the instructional design of the courses. In MOOCs on computer sciences, there are, for example, web-based Integrated Development Environments (IDEs) which can be integrated and that allow learners to do programming tasks directly in their browsers without installing desktop software. This work focuses on analyzing the effect on learners' engagement and behavior of integrating a third-party web-based IDE, Codeboard, in three MOOCs on Java programming with the purpose of promoting learning-by-doing (learning by coding in this case). In order to measure learners' level of engagement and behavior, data was collected from Codeboard on the number of compilations, executions and code generated, and compared between learners who registered in Codeboard to save and keep a record of their projects (registered learners) and learners who did not register in Codeboard and did not have access to these extra features (anonymous learners). The results show that learners who registered in Codeboard were more engaged than learners who did not register (in terms of number of compilations and executions), spent more time coding and did more changes in the base code provided by the teachers. The main implication of this study suggests the need for a trade-off between designing MOOCs that allow a very easy and anonymous access to external tools aimed for a more active learning, and forcing learners to give a step forward in terms of commitment in exchange for benefitting from additional features of the external tool used.