The assessment of procedural skills in physiotherapy education: a measurement study using the Rasch model

Abstract

Background: Procedural skills are a key element in the training of future physiotherapists. Procedural skills relate to the acquisition of appropriate motor skills, which allow the safe application of clinical procedures to patients. In order to evaluate procedural skills in physiotherapy education validated assessment instruments are required. Recently the assessment of procedural skills in physiotherapy education (APSPT) tool was developed. The overall aim of this study was to establish the structural validity of the APSPT. In order to do this the following objectives were examined: i) the fit of the items of APSPT to the Rasch-model, ii) the fit of the overall score to the Rasch model, iii) the difficulty of each test item and iv) whether the difficulty levels of the individual test items cover the whole capacity spectrum of students in pre-registration physiotherapy education. Methods: For this observational cross-sectional measurement properties study a convenience sample of 69 undergraduate pre-registration physiotherapy students of the HES-SO Valais-Wallis was recruited. Participants were instructed to perform a task procedure on a simulated patient. The performance was evaluated with the APSPT. A conditional maximum likelihood approach was used to estimate the parameters of a partial credit model for polytomous item responses. Item fit, ordering of thresholds, targeting and goodness of fit to the Rasch model was assessed. Results: Item fit statistics showed that 25 items of the APSPT showed adequate fit to the Rasch model. Disordering of item thresholds did not occur and the targeting of the APSPT was adequate to measure the abilities of the included participants. Undimensionality and subgroup homogeneity were confirmed. Conclusion: This study presented evidence for the structural validity of the APSPT. Undimensionality of the APSPT was confirmed and therefore presents evidence that the latent dimension of procedural skills in physiotherapy education consists of several subcategories. However, the results should be interpreted with caution given the small sample size.

Assessment of Physical Therapy Students Performance during Clinical Practice

“…As previous published literature has sufficiently documented the reliability and validity of APP assessment tool but not implemented this tool on DPT students, hence the results of this study cannot be matched with the result of the studies conducted in the past. The APP assessment tool seems to be very beneficial as it tests the multiple aspects to determine the clinical competency of a physical therapy student 10 .…”

Improving assessment of procedural skills in health sciences education: a validation study of a rubrics system in neurophysiotherapy

“…Table 3 presents the descriptive statistics of the students' scores on the final practical exam, as well as the difficulty (p) and discrimination (rpbis) indices of each rubric criteria. The scores range (3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15) practically covered the entire range of theoretical scores (0-15), with a median of 10. Students were distributed among the four proficiency levels considered as follows: 45 (31.7%) failed to reach the minimum level required to demonstrate basic mastery of the maneuvers (bellow basic level); 46 students (32.4%) showed the required level of mastery (basic level) and obtained an overall score in the final practical exam equal to or higher than 10 and lower than 12 point; 32 students (22%) obtained an overall score in the final practical exam equal to or higher than 12 and lower than 14 points, which is considered a proficient level; and 19 students (13.4%) were classified as advanced, having scored 14 points or higher in the final practical exam, which indicates a very high level of mastery of the maneuvers required in the subject.…”

“…However, acquiring and applying these skills in a fluent, automatized, and contextualized manner is more complex than expected [13]. In Physical Therapy, their acquisition requires the development of a large number and diversity of learning practices [14], as well as frequent and abundant feedback focused on the levels of execution shown by the students [15].…”

Refining assessment: Rasch analysis in health professional education and research

“…In addition to health professional education, there are broader applications of Rasch analysis in education, such as its use by international organisations like PISA, who designed the OECD's Programme for International Student Assessment (OECD, 2009). Rasch skills have many other important research applications, as they can be applied to a broad range of assessment domains, for example, the development of performance tests such as the Balance Intensity Scale, a clinical test of balance exercise performance (Farlie et al, 2019) or the Assessment of Procedural Skills in Physiotherapy, an assessment of procedural skill performance (Sattelmayer et al, 2020). Another example is the evaluation of competency to practise measured in the authentic workplace, such as the Assessment of Physiotherapy Practice (Dalton et al, 2011), which is now used in physiotherapy courses across Australia and New Zealand.…”