Advancing Consistency in Education: A Reliability Analysis of the Clinical Reasoning Assessment Tool

Abstract

Introduction. Clinical reasoning (CR) is a fundamental component of physical therapist practice in health care. The Clinical Reasoning Assessment Tool (CRAT) was developed to assess CR in Doctor of Physical Therapy (DPT) and residency education. However, the reliability of the CRAT has not been investigated. Our purpose was to investigate the interrater and intrarater reliability of the CRAT. Review of Literature. A range of assessment methods are available to assess CR, including didactic-based, simulation learning, and clinic-based assessments. The CRAT is designed to assess CR across learning environments. The CRAT assesses CR across 3 domains: (1) content knowledge, (2) procedural knowledge/psychomotor skills, and (3) conceptual reasoning. Subjects. Twenty-two faculty participated. Methods. Participants completed an electronic survey at 2 timepoints (T1, T2) that included training on use of the CRAT and 2 case videos of DPT students at different performance levels managing a standardized patient. Participants rated student performance with the CRAT for both case videos (T1) and repeated the sequence at least 30 days later (T2). Intraclass correlation coefficient (ICC) estimates and 95% confidence intervals were used to determine the interrater and intrarater reliability for each domain of the CRAT. Results. Video 1 was consistently rated higher for each CRAT domain (T1: 7.32–8.05; T2: 8.05–8.15) compared to video 2 (T1: 2.91–3.68; T2: 3.00–3.68). There were inverse relationships between CRAT ratings and number of years in DPT education, at an institution, and as a licensed physical therapist. At both timepoints, the interrater reliability for each domain was good to excellent (ICCs 0.60–0.79). The intrarater reliability was fair to excellent for all domains for each video (ICCs 0.45–0.75). Discussion and Conclusion. The CRAT is a reliable instrument for assessing DPT student CR during a simulated patient encounter. Implementation of the CRAT in DPT education may facilitate improved consistency of CR assessment across the learner continuum.