Effectiveness of Program Visualization: A Case Study with the ViLLE Tool

Program visualization is one of the various methods developed over the years to aid novices with their difficulties in learning to program. It consists of different graphical - often animated - and textual objects, visualizing the execution of programs. The aim of program visualization is to enhance students' understanding of different areas of program execution. Typical program visualization techniques include code highlighting, visualization of the call stack, and presenting information on variables. Despite the large number of studies performed on program visualization, little is known about the effects of such systems on learning. We have developed a program visualization tool called ViLLE, with the main objective of offering an environment for students to study the execution of example programs - whether written by students themselves or prepared by the teacher - and explore the changes in the program state data structures. A key feature of ViLLE is language independency, including parallel execution of a program in two different languages and the ability to define new languages. ViLLE also provides role information of program variables and supports the design and use of interactive pop-up questions. In this paper, we report and discuss the results of a study on the effectiveness of ViLLE. The research was conducted on university students in their first programming course. Students participated in a two hour session in a computer class, where they were randomly divided into two groups. The control group used only traditional textual material during the session, whereas for the treatment group, the same material was extended with interactive examples using ViLLE. With this research setting, we tried to answer two research questions: "Does ViLLE help students in learning to program?", and "Is there any difference in learning when previous programming experience is taken into account?" We found some support for a positive answer to the first question, although we couldn't fully reject the null hypothesis. For the second question, we obtained solid evidence that ViLLE enhances the learning of students with no prior programming experience substantially, so that the statistical differences between the novices and the more experienced learners disappeared as a result of a single training session. This indicates that program visualization indeed improves novice students' learning.