Permutation Tests in the Educational and Behavioral Sciences A Systematic Review

Over the past two decades, permutation tests (PTs) have received much attention in the educational and behavioral sciences. The aim of this article is to review the theoretical developments of PTs, the active areas in the educational and behavioral research using PTs, and the types of analysis under which PTs have been applied. We obtained 224 published articles, which included 141 theoretical articles and 83 application articles. After scrutinizing each article, we are happy to see that (1) some researchers began to advocate introducing PTs into basic statistics training; (2) computing load for PTs may be reduced dramatically by some intelligent algorithms; (3) PTs began to be applied in new areas such as studies on the relationship between brain and behavior and the relationship between gene and behavior; (4) besides simple types of analysis such as independent two-group comparison, PTs can also be carried out under more complex situations such as multivariate analysis. However, we should also notice that PTs are still mostly used for simple analyses (e. g., randomness analysis).