A threshold mixed-effects Tobit model for treatment-sensitive subgroup identification based on longitudinal measures with floor and ceiling effects and a continuous covariate

In the era of personalized medicine, there is an increasing interest in the identification of patients who may benefit from or be sensitive to a specific type of treatment. Recently a threshold linear mixed model was proposed to identify treatment-sensitive subgroups based on a continuous covariate when longitudinal measurements are the outcomes of the study. This model assumes, however, a normal distribution for these measurements. In some studies, the longitudinal measurements are restricted in an interval and subject to floor and ceiling effects caused by a portion of subjects with measurements on the boundaries of the interval, which would violate the normality assumption. In this paper, a threshold mixed-effects Tobit model is introduced to overcome this problem. The proposed models and inference procedures are assessed through simulation studies, as well as an application to the analysis of data from a randomized clinical trial.