Comparing two theories of grammatical knowledge assessment: a bifactor-MIRT analysis

This study compares two approaches to grammatical knowledge in language assessment: the structural view that regards grammatical knowledge as vocabulary and syntax (Bachman 1990), and the communicative view that perceives it as the binary combination of grammatical form and meaning (Purpura 2004). 1,491 second-year nursing students from eight medical colleges in China took a fifteen-item English grammar test (GT) that used retired items from the Language and Use section of the Public English Test System, Level Two. Data analysis comprised a series of dimensionality assessments (DAs) based on bifactor-multidimensional item response theory (bifactor-MIRT). This involved assessing the model fit achieved by structuring the GT tasks using a single grammatical factor, the structural approach and the communicative approach, and then comparing the relative performance of the three approaches. The results indicated that 1) despite its operational attraction, a unidimensional structure was insufficient to structure the GT tasks; 2) both the structural and the communicative approaches could sufficiently explain the underlying structure of the GT tasks; but 3) the communicative approach seemed to outperform the structural approach in uncovering the factual structure of the GT tasks. The study shows how bifactor-MIRT can be used to compare grammatical knowledge theories.