The effect of sampling variability on systems and individual speakers in likelihood ratio-based forensic voice comparison

The likelihood ratio (LR) framework has been widely adopted in voice (and other forensic) evidence evaluation. However, in developing any forensic comparison system, it is necessary to make subjective and pragmatic decisions, which in turn may affect the results that system produces. One such decision relates to not only the size of the samples used for training and testing the system, but also which specific individuals are used in the samples. The current study explores the relationship between sampling variability (i.e. the choice of speakers used for training and testing systems, rather than sample size) and the choice of linguistic features used. The first three formants and f0 from the vocalic portion of the filled pause um were used as input, as well as both vowel and nasal durations. 25 speakers were used in test, training and reference sets respectively. Experiments were carried out using all 31 logically possible combinations of features, and replicated 100 times using different configurations of 25 training and reference speakers. The results show that (a) overall, Cllr mean reduces with more features involved and no clear pattern is observed in Cllr range; meanwhile, considerable fluctuation is observed within individual speakers; (b) while the majority of speakers yield stronger mean LLRs in systems with three or more features, a few speakers can be well-separated using one or two features; (c) sampling variability in the training and reference speakers has limited effect on individual test speakers' LR outputs in same-speaker (SS) comparisons, but a marked effect on different-speaker (DS) LRs.