Matching on propensity and prognostic scores can lead to different estimates of heterogeneous treatment effects: a case study and simulation

The purpose of this study is to illustrate how matching approaches based on different balancing scores lead to variations in the treatment effect estimators. We introduced a case study evaluating the effect of anti-thrombotic agents on the severe progression of patients with intracerebral hemorrhage. We extracted subpopulations based on propensity and prognostic scores and then estimated the relative risk between the treatment groups. Furthermore, to illustrate the situation where the treatment effect estimates varied depending on employed balancing scores, we conducted a simulation experiment. In the case study, the matching using different balancing scores extracted subpopulations with different characteristics. Then, the estimated relative risk (95% confidence interval) was 1.27 (0.98–1.94) among the propensity score matched cohort, whereas it was 0.91 (0.76–1.08) among the prognostic score matched cohort. In the simulation experiments, the results indicated that the matching schemes based on different balancing scores created distinct matched cohorts, leading to varying estimates under treatment effect heterogeneity. Moreover, the variability of the estimated effect becomes substantial when there are covariates strongly related to the dependent variable of the nuisance functions. The difference in the selected subpopulation via matching based on different balancing scores is a thoughtful factor that can result in different estimates when there is effect heterogeneity. In practice, we recommend assessing the characteristics of the matched subpopulation and employing the balancing score that can estimate the treatment effect among the target population of interest in each study.