A Training Program Using an Audience Response System to Calibrate Dental Faculty Members Assessing Student Clinical Competence

Abstract

The aim of this study was to assess the effectiveness of calibration training of departmental faculty and competency graders using an audience response system on operative dentistry concepts across 12 months. The training sessions were designed to further solidify the process and equilibration of clinical opinions among faculty members and provide a more calibrated grading assessment during patient care for student performance feedback. Four (quarterly) calibration sessions occurred over 12 months in 2015. The first session was considered the baseline (control value) for this study. Pre-and post-calibration interrater agreement was assessed. Additionally, a pre and post assessment with ten Likert-scale questions was used to measure students' perceptions of instructional consistency. The results showed that a statistically significant increase in conceptual knowledge scores occurred for both departmental faculty members and competency graders across each of the four sessions (one-factor ANOVA; p<0.05). Interrater reliability agreement also significantly improved for both department faculty members and competency graders' clinical assessments over 12 months of implementation (Cohen's Kappa; p<0.05). There was a statistically significant increase in positive student perceptions on all ten questions (dependent t-test; p<0.05). Implementation of an audience response system for departmental and competency graders was found to be effective in facilitating a discussion forum, calibrating clinical assessments, and improving student perceptions. The positive results from this study support the value of dental schools' introducing faculty development programs to ensure consistent instruction for assessing dental student competence.Dr. Michael Metz is Vice Chairman,

Towards standardized clinical training: Developing an integrated clinical curriculum for dental trainees in a hospital of stomatology

“…Moreover, the personalities of clinical instructors are different from each other, and clinical instructor's different characteristics also affects clinical teaching effectiveness [11,12].Therefore, although there are personalized, group-style lectures in clinical training process, it is difficult to achieve complete standardized, unified clinical training effectiveness. It had revealed that the lack of standardization occurs, resulting in discrepancies among training effectiveness [13]. Thus, the unity of clinical training process is important in dental clinical education [14].…”

Dental Students’ Skills Assessments: Comparisons of Daily Clinical Grades and Clinical and Laboratory Assessments

“…Standardized rubrics are also likely to minimize problems with faculty calibration 10 . In an ideal setting, all clinical instructors would have participated in preclinical courses, which is perhaps the most efficient method of faculty calibration 11 . The reality, however, is that not all clinical faculty members have the opportunity to participate in preclinical courses.…”

“…When clinical instructors are not assigned to preclinical courses, a lack of standardization occurs, resulting in discrepancies among instructors. Discrepancies can exist in terminology, preparation design, and restoration design 11 . Faculty calibration is an essential process to allow more consistent evaluation with meaningful feedback to students’ performance both preclinically and clinically.…”

Assesment of Skill and Competence in Dental Education II: Assessment Methods

“…43 Fakültede kalibrasyon olmayışı; klinik derslerde mevcut olan ilave ve sık karşılaşılan bir sorundur. 44…”

“…İdeal olarak öğrencinin öğrenim seviyesi yükselirken, klinik eğitmenin kalibrasyon kazanmasına fırsat vermesi için, eğitmenin öğrencileri klinik ortamda değerlendirmeden önce preklinik derslerde görev alması gerekir. Klinik eğitmenler preklinik derslerde görev almayınca, standardizasyon yapılamaz ve eğitmenler arasında terminoloji, preparasyon ve restorasyon tasarımları açısından farklılıklar oluşur 44. Fakülte kalibrasyonu; öğrencilerin klinik ve preklinik performanslarında anlamlı geri bildirim oluşturarak, daha tutarlı değerlendirmeye imkan sağlar.Günlük klinik puanlamalar ve puanla değerledirmeler mukayese edildiğinde de çelişkiler ortaya çıkmaktadır.…”