Mixture modeling within the context of pharmacokinetic (PK)/pharmacodynamic (PD) mixed effects modeling is a useful tool to explore a population for the presence of two or more subpopulations, not explained by evaluated covariates. At present, statistical tests for the existence of mixed populations have not been developed. Therefore, a simulation study was undertaken to evaluate mixture modeling with NONMEM and explore the following questions. First, what is the probability of concluding that a mixed population exists when there truly is not a mixture (false positive significance level)? Second, what is the probability of concluding that a mixed population (two subpopulations) exists when there is truly a mixed population (power), and how well can the mixture be estimated, both in terms of the population parameters and the individual subject classification. Seizure count data were simulated using a Poisson distribution such that each subject's count could decrease from its baseline value, as a function of dose via an E-max model. The dosing design for the simulation was based on a trial with the investigational anti-epileptic drug pregabalin. Four hundred and forty seven subjects received pregabalin as add on therapy for partial seizures, each with a baseline seizure count and up to three subsequent seizure counts. For the mixtures, the two subpopulations were simulated to differ in their E-max values and relative proportions. One subpopulation always had its E-max set to unity (E-max hi), allowing the count to approach zero with increasing dose. The other subpopulation was allowed to vary in its E-max value (E-max lo = 0.75, 0.5, 0.25, and 0) and in its relative proportion (pr) of the population (pr = 0.05, 0.10, 0.25, and 0.50) giving a total of 4 . 4=16 different mixtures explored. Three hundred data sets were simulated for each scenario and estimations performed using NONMEM. Metrics used information about the parameter estimates, their standard errors (SE), the difference between minimum objective function (MOF) values for mixture and non-mixture models (MOF(delta)), the proportion of subjects classified correctly, and the estimated conditional probabilities of a subject being simulated as having E-max lo (E-max hi) given that they were estimated as having E-max lo (E-max hi) and being estimated as having E-max lo (E-max hi) given that they were simulated as having E-max lo (E-max hi). The false positive significance level was approximately 0.04 (using all 300 runs) or 0.078 (using only those runs with a successful covariance step), when there was no mixture. When simulating mixed data and for those characterizations with successful estimation and covariance steps, the median (range) percentage of 95% confidence intervals containing the true values for the parameters defining the mixture were 94% (89-96%), 89.5% (58-96%), and 95% (92-97%) for pr, E-max lo, and E-max hi, respectively. The median value of the estimated parameters pr, E-max lo (excluding the case when E-max lo was simulated to equal 0) and E-max hi within a scenario were within +/-28% of the true values. The median proportion of subjects classified correctly ranged from 0.59 to 0.96. In conclusion, when no mixture was present the false positive probability was less than 0.078 and when mixtures were present they were characterized with varying degrees of success, depending on the nature of the mixture. When the difference between subpopulations was greater (as E-max lo approached zero or pr approached 0.5) the mixtures became easier to characterize.
