Reliability of the Perceive, Recall, Plan and Perform System of Task Analysis: A criterion-referenced assessment

Objective: To conduct preliminary examination of the rater and test-reliability of the Perceive, Recall, Plan and Perform (PRPP) System of Task Analysis, an ecological measure designed to assess task-embedded information processing capacity during occupational therapy assessment of confused and agitated adults following traumatic brain injury. Methods: Occupational therapists observed and scored client performance using the PRPP System of Task Analysis. Correlational analysis and measures of agreement were performed to determine interrater and intrarater reliability. Test procedures were examined for reliability and internal consistency. Results: Interrater and test reliability considered three factors: therapists, clients and tasks. A moderate level of interrater reliability was achieved between trained therapists (intraclass correlation coefficient (ICC) = 0.60). Test procedures were highly reliable (ICC = 0.88). Across two measurement occasions, therapists showed a tendency towards harder rating on the second test occasion ((d) over bar -4.5%; 95% confidence interval for (d) over bar: -10.67% -> 3.17%). Conclusion: The findings of this study support the use of criterion-referenced tests in the area of occupational performance measurement. Occupational therapists achieved moderate interrater reliability when measuring the performance of adults with brain injury on various activities of daily living. Test procedures were found to be highly reliable in measuring the occupational performance of adults demonstrating confusion and agitation typical to the stage of post-traumatic amnesia following head injury.