Improving the quality of contextualized questions: an experimental investigation of focus

Setting examination questions in real-world contexts is widespread. However, when students are reading contextualized questions there is a risk that the cognitive processes provoked by the context can interfere with their understanding of the concepts in the question. Validity may then be compromised. We introduce the concept of focus: a question in a given context is focused to the extent that it addresses the aspects of the context that will be most salient in real life for the students being tested. A more focused context will then help activate relevant concepts, rather than interfering with comprehension and reasoning. In this study, the contexts of questions from a science test for 14-year-olds in England were manipulated to increase or decrease the amount of focus. In every instance more focused questions proved better than less focused ones. With additional evidence from interview protocols, we also consider the effects of context focus on the question answering process.