Intercoder Reliability for Use in Qualitative Research and Evaluation

Intercoder reliability can increase trustworthiness, accuracy, rigor, collaboration, and power sharing in qualitative research. Though not every qualitative design can utilize intercoder reliability, this article highlights how positivist qualitative research, community-based participatory research, and participatory evaluation all strengthen when intercoder reliability is sought among coding teams. Basic calculations for intercoder reliability can be completed manually or with software, making it an accessible metric to inform counseling research and evaluation, benefitting counselor education and practice. Interpreting metrics like percent agreement or kappa statistic largely depends on data context and potential impact of the study, which warrants researchers and evaluators to assess and understand the consequences of their work. Using ICR with clinical data such as interviews and other qualitative sources can be utilized to teach trainees the method in counseling research and evaluation and help make visible the important contributions of qualitative research using collaborative teams in counselor education. SIGNIFICANCE STATEMENT Intercoder reliability is a method in qualitative coding analysis that helps multiple coders increase accuracy in their results based on agreement in their coding. The method is particularly important in qualitative research and evaluation like positivist qualitative research, community-based participatory research, and participatory evaluation, where rigor, power sharing, and participation, respectively, are valued among the research teams.