Intra- and inter-rater reliability of the Assessment of Children's Hand Skills based on video recordings

Introduction: Hand skills are essential for children's occupational performance. The Assessment of Children's Hand Skills is a new assessment that utilizes naturalistic observations to capture children's actual hand-skill performance in everyday contexts. This study aimed to explore intra- and inter-rater reliability of the assessment based on video recordings, which are different from original naturalistic observations. Method: Two raters scored video recordings of 54 hand-skill activities performed by 12 children with developmental disabilities, twice in 2 weeks. Intra- and inter-rater reliability was examined at the individual hand-skill item scores, activity scores, and children's composite scores of the Assessment of Children's Hand Skills. Findings: Intra-rater reliability at item levels was generally acceptable, and both raters exhibited moderate to high agreement between the first and second evaluations (intraclass correlation coefficients = 0.61-0.93) at activity scores and children's composite scores. However, the agreement between the two raters was unacceptable for most hand-skill items and activity scores. After rater effects were adjusted by Rasch analysis in children's composite scores, the inter-rater reliability was improved (intraclass correlation coefficient = 0.81). Conclusion: This study shows that the Assessment of Children's Hand Skills based on video recordings is reliable within the same raters. Further research is required to confirm its inter-rater reliability by involving more training and raters with varied clinical experience.