Generalized linear mixed models for deception research: avoiding problematic data aggregation

While the concept of sampling variation is well understood by most researchers in the field of deception detection, previous studies have failed to account for the multiple sources of sampling variation present in typical experimental designs and use participant-level data as the dependant measure in analyses. These aggregated data, however, contain inherent biases that can mislead researchers. We argue that to appropriately test hypotheses and make inferences beyond a particular sample of participants, the decision-level data must be modelled directly. To illustrate how this can be achieved we provide an introduction to generalized linear mixed models (GLMMs) for the analysis of deception data and present Monte Carlo simulations demonstrating both the seriousness of the inherent biases present in participant-level data and the benefits of the GLMM approach. These simulations suggest that the empirical Type 1 and Type 2 error rates associated with main effects testing in deception research may be as high as 35% when data are aggregated by-judge' and as high as 60% when data are aggregated by-sender', respectively. When decision-level data are modelled directly, however, these rates are likely to be close to nominal levels (6% and 28%, respectively). Implications for past and future research are discussed.