Disparate Impact of Risk Assessment Instruments: A Systematic Review

Objective: One concern about the use of risk assessment instruments in legal decisions is the potential for disparate impact by race or ethnicity. This means that one racial or ethnic group will experience harsher legal outcomes than another because of higher or biased risk estimates. We conducted a systematic review of the literature to synthesize research examining the real-world impact of juvenile and adult risk instruments on racial/ethnic disparities in legal decision making. Hypotheses: Given the nature of research synthesis, we did not test formal hypotheses. Method: Our systematic literature search as of July 2023 identified 21 articles that investigated the disparate impact of 13 risk assessment instruments on various legal outcomes. Most of these instruments were actuarial pretrial screening instruments. Results: Our narrative synthesis indicated that there is not strong evidence of risk instruments contributing to greater system disparity. Ten articles indicated that adopting risk instruments did not create (or exacerbate preexisting) disparities, and eight articles found that instrument use reduced disparities in legal decision making. Three articles reported evidence of disparate impact of risk instruments; only one of these studies received a strong study quality assessment score. We observed a scarcity of high-quality articles that employed what we deem to be the gold standard approach for examining the disparate impact of risk instruments (i.e., pretest-posttest design). Conclusions: The evidence signals that risk instruments can contribute to reductions in disparities across multiple stages of legal decision making. Yet study quality remains low, and most research has been conducted on decisions during the pretrial stage. More rigorous research on disparate impact across diverse legal decision points and approaches to risk assessment is needed.