Threats to external validity in the neuroprediction of substance use treatment outcomes

Background: Tools predicting individual relapse risk would invaluably inform clinical decision-making (e.g. level-of-care) in substance use treatment. Studies of neuroprediction - use of neuromarkers to predict individual outcomes - have the dual potential to create such tools and inform etiological models leading to new treatments. However, financial limitations, statistical power demands, and related factors encourage restrictive selection criteria, yielding samples that do not fully represent the target population. This problem may be further compounded by a lack of statistical optimism correction in neuroprediction research, resulting in predictive models that are overfit to already-restricted samples. Objectives: This systematic review aims to identify potential threats to external validity related to restrictive selection criteria and underutilization of optimism correction in the existing neuroprediction literature targeting substance use treatment outcomes. Methods: Sixty-seven studies of neuroprediction in substance use treatment were identified and details of sample selection criteria and statistical optimism correction were extracted. Results: Most publications were found to report restrictive selection criteria (e.g. excluding psychiatric (94% of publications) and substance use comorbidities (69% of publications)) that would rule-out a considerable portion of the treatment population. Furthermore, only 21% of publications reported optimism correction. Conclusion: Restrictive selection criteria and underutilization of optimism correction are common in the existing literature and may limit the generalizability of identified neural predictors to the target population whose treatment they would ultimately inform. Greater attention to the inclusivity and generalizability of addiction neuroprediction research, as well as new opportunities provided through open science initiatives, have the potential to address this issue.