Score generalizability of academic writing tasks: Does one test method fit it all?

Generalizability of writing scores has always been a longstanding concern in L2 writing assessment. A number of studies have been conducted to investigate this topic during the last two decades. However, with the introduction of new test methods, such as reading-to-write tasks, generalizability studies need to focus on the score accuracy of different task types. The current study attempts to examine how reading-to-write tasks are comparable to independent tasks in terms of score generalizability. To look into this issue, 115 Egyptian university students were asked to write on two independent tasks and two reading-to-write tasks. The essays were holistic ally scored by three raters, who received training prior to scoring. The study design followed a fully crossed univariate design (p x t x r), and consequently GENOVA (Crock & Brennan, 1983) was used to analyze the results. Results showed that the reading-to-write tasks yielded as reliable scores as independent tasks. In addition, the result indicated that score generalizability is very low when using one writing task due to the large (pt) variance component. Implications and limitations of the study are presented.