Pharmacy Student Perceptions and Preferences of In-person Versus Video-Recorded Evaluations in Skills-Based Courses

Abstract

Objective. To determine pharmacy students' preferences and perceptions of in-person and video evaluations. The relationship between student perceptions and academic achievement has been established, but little research on student perceptions of evaluation methods exists. Methods. A mixed methods survey was administered to 447 first-, second-, and third-professional year pharmacy students enrolled in one public United States Doctor of Pharmacy program. Fourteen 5-point Likert-type scale quantitative items and four qualitative items measured student perceptions. Eight response choice items measured preferences. Paired t-tests compared perceptions; independent t-tests compared perceptions between students exposed and not exposed to video evaluations. Two researchers performed thematic content analysis of the qualitative responses. Results. Students (n=444, 99.3% response rate) perceived in-person and video evaluations as significantly different, with in-person evaluations being perceived more positively on all items except for nervousness. Students exposed to video felt significantly more positive towards video evaluations than video-naïve students on nine items but felt significantly less positive towards video evaluations in terms of quality (Δ=1.24 vs. 0.83) and amount (Δ=1.14 vs 0.77) of written feedback. Students valued interactions with a more diverse pool of evaluators afforded by video evaluations but did not view the video technology as applicable to future practice. Conclusion.Students viewed in-person evaluations significantly more positively than video evaluations. This effect is mitigated by exposure to video, suggesting that concerns regarding video evaluations are based on conjecture rather than experience. This study highlights the need to reduce technological issues and improve written feedback associated with video evaluations.

Viability of Virtual Skills-based Assessments Focused on Communication

“…These perceptions were similar to related work conducted before the pandemic. 18 Since our study was conducted at the beginning of the pandemic, similarity would be expected. Perceptions may continue to shift as familiarity with video-based distance instruction improves.…”

“…14,15 Limited data on faculty and student perceptions of skills-based assessments is found in medical and pharmacy literature and draws varying conclusions of satisfaction. 4,[16][17][18] The purpose of this project was to evaluate faculty and student perceptions and student performance of virtual skills-based assessments compared to in-person skills-based assessments focused on communication.…”

Feedback for Learning in Pharmacy Education: A Scoping Review

“…The moderate theory talk articles examined new didactic courses and course interventions, and included feedback on student performance within their design, but did not assess the given feedback for quality or impact. Overall, as a group, these papers did not focus on feedback as a key study objective or outcome; only four studies included feedback in their study objectives, but no aspect of the feedback provided was included for analysis [ 14 , 15 , 16 , 17 ]. These four papers tested the use of feedback in three different areas: improving CPR skills development [ 14 ]; subjective, objective, assessment, and plan (SOAP) note writing [ 15 ]; and student performance on verbal competency and patient counseling and interviewing assessments [ 16 , 17 ].…”

“…Overall, as a group, these papers did not focus on feedback as a key study objective or outcome; only four studies included feedback in their study objectives, but no aspect of the feedback provided was included for analysis [ 14 , 15 , 16 , 17 ]. These four papers tested the use of feedback in three different areas: improving CPR skills development [ 14 ]; subjective, objective, assessment, and plan (SOAP) note writing [ 15 ]; and student performance on verbal competency and patient counseling and interviewing assessments [ 16 , 17 ]. The other studies’ feedback focus areas were patient communication including counseling and other simulated patient interactions [ 18 , 19 , 20 , 21 , 22 ]; patient work up including SOAP notes [ 18 , 23 , 24 , 25 , 26 , 27 , 28 ] or oral case presentations and care plans [ 26 , 29 , 30 , 31 ]; and other topics such as evidence appraisal and research skills [ 32 , 33 ]; self-assessments [ 34 ]; disease information [ 35 ]; pharmacy calculations [ 36 ]; and pharmacology experiments [ 37 ].…”

Changes to summative skills‐based assessments within the Big Ten Academic Alliance Performance‐Based Assessment Collaborative (BTAA‐PBAC) due to COVID‐19

“…Students often cited that the distance between their evaluator hindered their ability to establish a connection or professional rapport with their evaluator. 16 Further, practice of skills is often required to achieve near mastery of a skill, which for physical assessment and other in-person skills, would likely be difficult to achieve in a virtual environment. 17 To address this lack of information in the skills-based laboratory setting, members of the Big Ten Academic Alliance-Performance Based Assessment Collaborative (BTAA-PBAC) developed this study.…”