Examining the normality assumption of a design-comparable effect size in single-case designs

被引:3
作者
Chen, Li-Ting [1 ]
Chen, Yi-Kai [2 ]
Yang, Tong-Rong [2 ]
Chiang, Yu-Shan [3 ]
Hsieh, Cheng-Yu [2 ,4 ]
Cheng, Che [2 ]
Ding, Qi-Wen [5 ]
Wu, Po-Ju [6 ]
Peng, Chao-Ying Joanne [2 ,6 ]
机构
[1] Univ Nevada, Dept Educ Studies, Reno, NV 89557 USA
[2] Natl Taiwan Univ, Dept Psychol, Taipei, Taiwan
[3] Indiana Univ Bloomington, Dept Curriculum & Instruct, Bloomington, IN USA
[4] Univ London, Royal Holloway, Dept Psychol, Egham, England
[5] Acad Sinica, Inst Sociol, Taipei, Taiwan
[6] Indiana Univ Bloomington, Dept Counseling & Educ Psychol, Bloomington, IN USA
关键词
Single-case; Intervention; Standardized mean difference; Effect size; Design comparable; Normality; MULTIPLE-BASE-LINE; DIFFERENCE EFFECT SIZE; MONTE-CARLO; MAXIMUM-LIKELIHOOD; MULTILEVEL MODELS; SUBJECT RESEARCH; INTERVENTION; SAMPLE; METAANALYSIS; VIOLATIONS;
D O I
10.3758/s13428-022-02035-8
中图分类号
B841 [心理学研究方法];
学科分类号
040201 ;
摘要
What Works Clearinghouse (WWC, 2022) recommends a design-comparable effect size (D-CES; i.e., g(AB)) to gauge an intervention in single-case experimental design (SCED) studies, or to synthesize findings in meta-analysis. So far, no research has examined g(AB)'s performance under non-normal distributions. This study expanded Pustejovsky et al. (2014) to investigate the impact of data distributions, number of cases (m), number of measurements (N), within-case reliability or intra-class correlation (rho), ratio of variance components (lambda), and autocorrelation (phi) on g(AB) in multiple-baseline (MB) design. The performance of g(AB) was assessed by relative bias (RB), relative bias of variance (RBV), MSE, and coverage rate of 95% CIs (CR). Findings revealed that g(AB) was unbiased even under non-normal distributions. g(AB)'s variance was generally overestimated, and its 95% CI was over-covered, especially when distributions were normal or nearly normal combined with small m and N. Large imprecision of g(AB) occurred when m was small and rho was large. According to the ANOVA results, data distributions contributed to approximately 49% of variance in RB and 25% of variance in both RBV and CR. m and rho each contributed to 34% of variance in MSE. We recommend g(AB) for MB studies and meta-analysis with N >= 16 and when either (1) data distributions are normal or nearly normal, m = 6, and rho = 0.6 or 0.8, or (2) data distributions are mildly or moderately non-normal, m >= 4, and rho = 0.2, 0.4, or 0.6. The paper concludes with a discussion of g(AB)'s applicability and design-comparability, and sound reporting practices of ES indices.
引用
收藏
页码:379 / 405
页数:27
相关论文
共 97 条
[41]   Single-Case Intervention Research Design Standards [J].
Kratochwill, Thomas R. ;
Hitchcock, John H. ;
Horner, Robert H. ;
Levin, Joel R. ;
Odom, Samuel L. ;
Rindskopf, David M. ;
Shadish, William R. .
REMEDIAL AND SPECIAL EDUCATION, 2013, 34 (01) :26-38
[42]   Coaching via Telehealth: Caregiver-Mediated Interventions for Young Children on the Waitlist for an Autism Diagnosis Using Single-Case Design [J].
Kunze, Megan G. ;
Machalicek, Wendy ;
Wei, Qi ;
St Joseph, Stephanie .
JOURNAL OF CLINICAL MEDICINE, 2021, 10 (08)
[43]   The Effect of a Tier 2 Multicomponent Fraction Intervention for Fifth Graders Struggling With Fractions [J].
Lee, Jihyun ;
Bryant, Diane Pedrotty ;
Bryant, Brian R. .
REMEDIAL AND SPECIAL EDUCATION, 2023, 44 (01) :28-42
[45]   The influence of violations of assumptions on multilevel parameter estimates and their standard errors [J].
Maas, CJM ;
Hox, JJ .
COMPUTATIONAL STATISTICS & DATA ANALYSIS, 2004, 46 (03) :427-440
[46]   Commentary on the What Works Clearinghouse Standards and Procedures Handbook (v. 4.1) for the Review of Single-Case Research [J].
Maggin, Daniel M. ;
Barton, Erin ;
Reichow, Brian ;
Lane, Kathleen ;
Shogren, Karrie A. .
REMEDIAL AND SPECIAL EDUCATION, 2022, 43 (06) :421-433
[47]   Effects of Compounded Nonnormality of Residuals in Hierarchical Linear Modeling [J].
Man, Kaiwen ;
Schumacker, Randall ;
Morell, Monica ;
Wang, Yurou .
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 2022, 82 (02) :330-355
[48]   Nonlinear Growth Models as Measurement Models: A Second-Order Growth Curve Model for Measuring Potential [J].
McNeish, Daniel ;
Dumas, Denis .
MULTIVARIATE BEHAVIORAL RESEARCH, 2017, 52 (01) :61-85
[49]   Nonparametric meta-analysis for single-case research: Confidence intervals for combined effect sizes [J].
Michiels, Bart ;
Onghena, Patrick .
BEHAVIOR RESEARCH METHODS, 2019, 51 (03) :1145-1160
[50]  
Mioevic M, 2020, Small sample size solutions: A guide for applied researchers and practitioners, P87, DOI [10.4324/9780429273872, DOI 10.4324/9780429273872]