Implicit Bias and the Association of Redaction of Identifiers With Residency Application Screening Scores

This quality improvement study evaluates the association of redaction of applicant identifiers with review scores on ophthalmology residency applications as 1 intervention to address implicit bias. Question Is the redaction of identifiers that may trigger implicit bias (eg, name, sex or gender, race and ethnicity) associated with differences in scores on ophthalmology residency application screening for applicants underrepresented in medicine? Findings This quality improvement study found that redaction of identifiers was not associated with differences in application scores. The distribution of application scores was similar for redacted vs unredacted applications, with no difference based on an applicant's sex, underrepresentation in medicine status (traditionally comprising American Indian or Alaskan Native, Black, and Hispanic individuals), or international medical graduate status. Meaning Although study size may have precluded identifying differences, these findings suggest that redacting applicant characteristics is not associated with application scores among disadvantaged candidates. Importance Diversity in the ophthalmology profession is important when providing care for an increasingly diverse patient population. However, implicit bias may inadvertently disadvantage underrepresented applicants during resident recruitment and selection. Objective To evaluate the association of the redaction of applicant identifiers with the review scores on ophthalmology residency applications as an intervention to address implicit bias. Design, Setting, and Participants In this quality improvement study, 46 faculty members reviewed randomized sets of 462 redacted and unredacted applications from a single academic institution during the 2019-2020 ophthalmology residency application cycle. Interventions Applications electronically redacted for applicant identifiers, including name, sex or gender, race and ethnicity, and related terms. Main Outcomes and Measures The main outcome was the distribution of scores on redacted and unredacted applications, stratified by applicant's sex, underrepresentation in medicine (URiM; traditionally comprising American Indian or Alaskan Native, Black, and Hispanic individuals) status, and international medical graduate (IMG) status; the application score beta coefficients for redaction and the applicant and reviewer characteristics were calculated. Applications were scored on a scale of 1 to 9, where 1 was the best score and 9 was the worst score. Scores were evaluated for a significant difference based on redaction among female, URiM, and IMG applicants. Linear regression was used to evaluate the adjusted association of redaction, self-reported applicant characteristics, and reviewer characteristics with scores on ophthalmology residency applications. Results In this study, 277 applicants (60.0%) were male and 71 (15.4%) had URiM status; 32 faculty reviewers (69.6%) were male and 2 (0.4%) had URiM status. The distribution of scores was similar for redacted vs unredacted applications, with no difference based on sex, URiM status, or IMG status. Applicant's sex, URiM status, and IMG status had no association with scores in multivariable analysis (sex, beta = -0.08; 95% CI, -0.32 to 0.15; P = .26; URiM status, beta = -0.03; (95% CI, -0.36 to 0.30; P = .94; and IMG status, beta = 0.39; 95% CI, -0.24 to 1.02; P = .35). In adjusted regression, redaction was not associated with differences in scores (beta = -0.06 points on a 1-9 scale; 95% CI, -0.22 to 0.10 points; P = .48). Factors most associated with better scores were attending a top 20 medical school (beta = -1.06; 95% CI, -1.37 to -0.76; P < .001), holding an additional advanced degree (beta = -0.86; 95% CI, -1.22 to -0.50; P < .001), and having a higher United States Medical Licensing Examination Step 1 score (beta = -0.35 per 10-point increase; 95% CI, -0.45 to -0.26; P < .001). Conclusions and Relevance This quality improvement study did not detect an association between the redaction of applicant characteristics on ophthalmology residency applications and the application review scores among underrepresented candidates at this institution. Although the study may not have been powered adequately to find a difference, these findings suggest that the association of redaction with application review scores may be preempted by additional approaches to enhance diversity, including pipeline programs, implicit bias training, diversity-centered culture and priorities, and targeted applicant outreach. Programs may adapt this study design to probe their own application screening biases and track over time before-and-after bias-related interventions.