Video Recording in Veterinary Medicine OSCEs: Feasibility and Inter-rater Agreement between Live Performance Examiners and Video Recording Reviewing Examiners

Tan, Jean-Yin; Hunt, Julie A.; Kwong, Grace P. S.; Farrell, Robin; Bell, Catriona; Read, Emma K.

doi:10.3138/jvme-2019-0142

Cited by 12 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Raters in this study reported they spent a considerable amount of time rating the videos and welcomed the prescribed break after rating four recordings. Similar findings were reported by Tan et al (2020) in a study evaluating rating of digitally recorded OSCE stations 65 . Digitally recorded evaluations of surgical skills have also been assessed for evidence of reliability in other veterinary studies, 20,23,26,52,66 so this remains a feasible option.…”

Section: Discussionsupporting

confidence: 70%

“…Similar findings were reported by Tan et al (2020) in a study evaluating rating of digitally recorded OSCE stations. 65 Digitally recorded evaluations of surgical skills have also been assessed for evidence of reliability in other veterinary studies, 20,23,26,52,66 so this remains a feasible option. While the digital recordings in this study were used solely to collect data for the reliability study, digital recordings can be a powerful tool to provide feedback to students on their performance to help them enhance their proficiency and will be considered for this purpose in the future.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment

Farrell

Gilbert²,

Betance

et al. 2022

Veterinary Surgery

Self Cite

View full text Add to dashboard Cite

To gather and evaluate validity evidence in the form of content and reliability of scores produced by 2 surgical skills assessment instruments, 1) a checklist, and 2) a modified form of the Objective Structured Assessment of Technical Skills (OSATS) global rating scale (GRS).Study design: Prospective randomized blinded study. Sample population: Veterinary surgical skills educators (n =10) evaluated content validity. Scores from students in their third preclinical year of veterinary school (n = 16) were used to assess reliability.Methods: Content validity was assessed using Lawshe's method to calculate the Content Validity Index (CVI) for the checklist and modified OSATS GRS.The importance and relevance of each item was determined in relation to skills needed to successfully perform supervised surgical procedures. The reliability of scores produced by both instruments was determined using generalizability (G) theory. Results: Based on the results of the content validation, 39 of 40 checklist items were included. The 39-item checklist CVI was 0.81. One of the 6 OSATS GRS items was included. The 1-item GRS CVI was 0.80. The G-coefficients for the 40-item checklist and 6-item GRS were 0.85 and 0.79, respectively. Conclusion:Content validity was very good for the 39-item checklist and good for the 1-item OSATS GRS. The reliability of scores from both instruments was acceptable for a moderate stakes examination. Impact: These results provide evidence to support the use of the checklist described and a modified 1-item OSAT GRS in moderate stakes examinations when evaluating preclinical third-year veterinary students' technical surgical skills on low-fidelity models.

show abstract

Section: Discussionsupporting

confidence: 70%

Section: Discussionmentioning

confidence: 99%

Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment

Farrell

Gilbert²,

Betance

et al. 2022

Veterinary Surgery

Self Cite

View full text Add to dashboard Cite

show abstract

“…64 To date, the OSCE been widely used in veterinary education, including the evaluation of surgical skills, sometimes mixed with nonsurgical clinical skills. [65][66][67][68] Objective structured clinical examinations are multistation, multirater assessments where the student rotates between stations and raters, being scored on discrete tasks rather than entire procedures. 69 Reliability of an OSCE is dependent upon having an adequate number of stations, rater training if multiple raters are used to score the same station, and good-quality rubrics.…”

Section: Forms Of Surgical Assessmentmentioning

confidence: 99%

“…While in-person scoring of OSCE and surgical performance typically allows the best visualization of the trainee performing the task, having raters score video recorded OSCE stations has proven feasible and reliable, although it did not reduce the time required for grading and there were some differences between examiners in pass/fail decisions at individual stations. 67 Scoring other surgical skills examinations using video recordings has also generated reliable results. 78,86 GoPro video cameras have been used, either head mounted 95 or attached to a surgical light, 86,96 to capture surgical videos successfully.…”

Section: Video Recorded Assessmentsmentioning

confidence: 99%

“…To date, the OSCE been widely used in veterinary education, including the evaluation of surgical skills, sometimes mixed with nonsurgical clinical skills 65–68 . Objective structured clinical examinations are multistation, multirater assessments where the student rotates between stations and raters, being scored on discrete tasks rather than entire procedures 69 .…”

Section: How Can Surgical Skills Be Assessed?mentioning

confidence: 99%

See 1 more Smart Citation

What's the evidence? A review of current instruction and assessment in veterinary surgical education

2022

Self Cite

View full text Add to dashboard Cite

Veterinary surgical education is improved by studying the relevant literature.The aim of this literature review was to report the theory and methods used to provide surgical education to veterinary students; to discuss the training most likely to create a competent general practitioner; and to review assessment methods for simulated and live surgeries. The literature reviewed demonstrates that new graduates are expected to perform small animal sterilization, dental extraction, onychectomy, abscess treatment, aural hematoma repair, and mass removal with little or no supervision. Students require repetitive practice to reach competence; some practice can take place in a clinical skills laboratory on models or cadavers. When training novices, distributing practice over a longer time improves retention, but months without practice causes skills to decay.Suturing skills may be taught on models at a ratio of 1 instructor per 10 students. Veterinary students require 6-10 repetitions of each small animal sterilization surgery to reach competence; however, learning curves for other surgeries have not been established. Assessment of surgical skills has been undertaken using objective structured clinical examinations (OSCEs) and observed model and live surgeries. Additional validated rubrics are needed in the field. Surgical educators are urged to continue to search for innovative ways to foster deliberate practice, assess skill, and provide feedback for students.

show abstract

Trained to cut? A literature review of veterinary surgical resident training

Simons,

Hunt,

Anderson

2024

Veterinary Surgery

Self Cite

View full text Add to dashboard Cite

A surgical residency trains veterinary graduates to a higher level of expertise in surgical procedures than is possible during veterinary school and prepares a resident to pursue board certification in surgery. The education of veterinary surgical residents has changed minimally since its inception in the twentieth century, and there are insufficient studies to determine if residency programs are producing surgeons with competence in each of the necessary procedural categories. The aims of this review were to report the current theory and methods used to provide surgical education to residents, to discuss the training most likely to create a competent, board‐certified surgeon and to review assessment methods used during training. Several literature searches using broad terms such as “veterinary surgery residency,” “veterinary surgery resident,” and “veterinary surgical training” were performed using PubMed, CAB abstracts, and Google Scholar. Literature pertinent to theory, methods, training, and assessment of veterinary surgical residents was included. The reviewed literature demonstrated the need for research‐based learning curves for specific procedures. Simulation training is known to facilitate deliberate practice and should be leveraged where possible to reach competency. The creation of validated assessment methods should be pursued as it enables assessment of competency instead of inferring its development from case logs. Understanding and supporting learner cognition and providing sufficient feedback remain important issues in the field. Surgical educators are urged to continue to search for innovative and evidence‐based ways to train competent surgical residents.

show abstract

Video Recording in Veterinary Medicine OSCEs: Feasibility and Inter-rater Agreement between Live Performance Examiners and Video Recording Reviewing Examiners

Cited by 12 publications

References 19 publications

Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment

Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment

What's the evidence? A review of current instruction and assessment in veterinary surgical education

Trained to cut? A literature review of veterinary surgical resident training

Contact Info

Product

Resources

About