Reference and target sample age distribution impacts between model types in dental developmental age estimation

The performance of age estimation methods may vary due to a combination of method- and sample-related factors. Method development and evaluation necessitates understanding what influences these factors have on age estimation outcomes. In the specific context of juvenile dental age estimation, we used a single dataset and complete factorial design to systematically test four potential sources of difference: age distributions of reference and target sample (uniform, unimodal, U-shaped), Bayesian (multivariate Bayesian cumulative probit) vs. classical regression modeling (multivariate adaptive regression splines i.e. MARS), and model selection bias. The dataset consisted of 850 sets of left mandibular molar scores from London children 5–18 years old. True age and estimated age intervals in target samples were compared for bias, root-mean-squared error, precision, and accuracy using locally weighted smoothing of performance measures across the age range and means of performance metrics between factor-level combinations. We found interactions of model type, reference distribution, and target distribution. MARS models showed consistent evidence of age mimicry. Central tendency of the reference sample corresponded with increased bias while central tendency of the target sample corresponded with reduced RMSE and reduced precision for both model types. We found evidence of model selection bias, mitigated through averaging model metrics. We conclude that reference and target sample distribution influences and model selection bias are sufficient to cause difference in model performance within a single population. We suggest using Bayesian modeling, drawing uniform reference and target samples, and calculating test error on a hold-out sample to mitigate these challenges in method development.