The paradox of peer review: Admitting too much or allowing too little?

Peer review - the means by which one's equals assess the quality of one's scholarly work - has been used to determine academic merit for more than three centuries. Although the results of academic peer reviews are frequently challenged, peer review continues to be a commonplace of academic life. In the present scholarship climate, challenges to peer review come primarily from two sides. On one side there are those who believe that high academic standards are not being upheld by peer review panels, thus admitting weak proposals, people, programs, and articles. On the other side there are those who believe that review panels uphold conventional standards, thus disqualifying innovative projects and worthy people with diverse perspectives. In this article, I take up these two challenges. Together, they reveal a paradox of the current system of peer review: Both sides are right. And, the paradox has led to increased fragmentation within the field of educational research. Yet, despite the limitations of peer review, no viable alternatives exist. Thus, I will argue that the best we can do is to re-consider what we mean by "merit" in educational research, and to train new educational researchers in ways that expand their ability to judge merit broadly.