Pre-test influences on the effectiveness of digital-game based learning: A case study of a fire safety game

In recent years, critiques have been formulated regarding current evaluation methods of digital game-based learning (DGBL) effectiveness, raising doubt with regard to the validity of certain results. A major issue of contention is whether or not a pre-test should be administered, gauging for baseline measures of knowledge that are targeted using an educational intervention. The present study aims to explore the advantages and disadvantages of adding a pre-test in DGBL effectiveness research. For this purpose, an effectiveness study of a fire safety training in a hospital was conducted using a Solomon four-group design. The experimental groups received a game-based intervention (N = 65) of which one group received a pre and a post-test (n = 3 4) and one group only received a post-test (n = 31). The control groups received traditional classroom instruction (n = 68), of which one group received a pre -and a post-test (n = 39) and one group only received a post-test (n = 29). No main effect of testing was found. However, an interaction effect between pre-test and intervention was detected. Subjects who have received a pre-test in this group score significantly higher (p < 0.05) on the post-test than subjects in the traditional classroom group who did not receive a pre-test. This was not the case in the game group. When the administration of a pre-test influences the control group's receptivity to the intervention, but not that of the experimental group, results of an effectiveness study may be biased. Hence comparison of post-test scores of different treatments in pre-test/post-test designs may be problematic. This is an important finding in the context of DGBL effectiveness research as the presence of a pre-test may artificially inflate the learning outcomes of the control condition. Therefore, further research should take this into account and look for possible solutions to solve this discrepancy. However, in the present study, we were able to show that the game was highly effective, as both game groups still outperformed the slide-based group that received a pre-test. The Solomon four group design has thus shown its added value and more effectiveness studies on DGBL implementing this design are required in order to further validate these results. (C) 2017 Published by Elsevier Ltd.