Evaluating whether the proportional odds models to analyse ordinal outcomes in COVID-19 clinical trials is providing clinically interpretable treatment effects: A systematic review

Background: After an initial recommendation from the World Health Organisation, trials of patients hospitalised with COVID-19 often include an ordinal clinical status outcome, which comprises a series of ordered categorical variables, typically ranging from 'Alive and discharged from hospital' to 'Dead'. These ordinal outcomes are often analysed using a proportional odds model, which provides a common odds ratio as an overall measure of effect, which is generally interpreted as the odds ratio for being in a higher category. The common odds ratio relies on the assumption of proportional odds, which implies an identical odds ratio across all ordinal categories; however, there is generally no statistical or biological basis for which this assumption should hold; and when violated, the common odds ratio may be a biased representation of the odds ratios for particular categories within the ordinal outcome. In this study, we aimed to evaluate to what extent the common odds ratio in published COVID-19 trials differed to simple binary odds ratios for clinically important outcomes. Methods: We conducted a systematic review of randomised trials evaluating interventions for patients hospitalised with COVID-19, which used a proportional odds model to analyse an ordinal clinical status outcome, published between January 2020 and May 2021. We assessed agreement between the common odds ratio and the odds ratio from a standard logistic regression model for three clinically important binary outcomes: 'Alive', 'Alive without mechanical ventilation', and 'Alive and discharged from hospital'. Results: Sixteen randomised clinical trials, comprising 38 individual comparisons, were included in this study; of these, only 6 trials (38%) formally assessed the proportional odds assumption. The common odds ratio differed by more than 25% compared to the binary odds ratios in 55% of comparisons for the outcome 'Alive', 37% for 'Alive without mechanical ventilation', and 24% for 'Alive and discharged from hospital'. In addition, the common odds ratio systematically underestimated the odds ratio for the outcome 'Alive' by -16.8% (95% confidence interval: -28.7% to -2.9%, p = 0.02), though differences for the other outcomes were smaller and not statistically significant (-8.4% for 'Alive without mechanical ventilation' and 3.6% for 'Alive and discharged from hospital'). The common odds ratio was statistically significant for 18% of comparisons, while the binary odds ratio was significant in 5%, 16%, and 3% of comparisons for the outcomes 'Alive', 'Alive without mechanical ventilation', and 'Alive and discharged from hospital', respectively. Conclusion: The common odds ratio from proportional odds models often differs substantially to odds ratios from clinically important binary outcomes, and similar to composite outcomes, a beneficial common OR from a proportional odds model does not necessarily indicate a beneficial effect on the most important categories within the ordinal outcome.