Intra‐ and Inter‐Examiner Variability in Evaluating Preclinical Pediatric Dentistry Operative Procedures

Abstract

Many investigators have reported attempts to develop reliable laboratory and clinic evaluation systems. However, few studies, regardless of level of success, have used an analytic procedure to identify those components of the evaluation system that, if reined further, could improve reliability. The purpose of this study was to compare intra-and inter-examiner variability in two evaluation methods: glance and grade (global), and checklist and criteria (analytical). Three faculty staff members with more than ten years of clinical and teaching experience evaluated operative procedures performed on plastic teeth representing the primary teeth by thirty dental students in pediatric dentistry preclinical laboratory sessions. The preparations were graded blindly by each of the three evaluators (A, B, and C) three times without magniication. The values were statistically analyzed using Wilcoxon signed rank test and Friedman test setting value of signiicance at 5 percent. The study revealed that, among the three examiners, the intra-examiner variability was nonsigniicant in most situations. On the other hand, there was statistically signiicant variability between evaluators (i.e., inter-examiner) for almost all preparations. Neither cutting off the scores nor using either evaluation method (glance and grade or criteria and checklist) caused an improvement in variability. The problem of inter-examiner reliability and variability still existed.

Undergraduate dental students’ perspective on the implementation of digital dentistry in the preclinical curriculum: a questionnaire survey

“…Renne et al reported that students often perceive assessments as subjective and arbitrary [16]. This perception may lead to decreased self-awareness and may negatively influence students' performance [17]. Taken together, at the moment each university seems to develop an individual concept.…”

Assesment of Skill and Competence in Dental Education II: Assessment Methods

“…Diş hekimliği fakültesi öğrencilerini klinikte değerlendirirken kullanılan sistemler analiz edilmekte; formların ve müfredatın hangi bölümlerinin yeniden düzenlenmesi ve iyileştirilmesi gerektiği araştırılmaktadır. [13][14][15][16][17][18][19][20] Klinik becerilerin güvenilir, doğru ve kapsamlı şekilde değerlendirilebilmesi için birçok sistem vardır. [12][13][14] Önceleri, klinik uygulamalar basitçe geçme/kalma şeklinde sonuç bazlı değerlendirilir ve başarısız öğrenciler uygulamalarını tekrarlamak zorunda kalırdı.…”

“…15 Evet/hayır cevaplı veya oran skalası içeren, uygulama sürecine göre yapılandırılan kontrol listelerinin spesifik klinik uygulamalar için güvenilir, doğru ve uygulanabilir olduğu ileri sürülürken, [16][17][18] kontrol listelerinin kullanımının pedodonti preklinik uygulamalarının değerlendirilmesinde basit olarak "gözle bakarak puanlama" ya üstün olmadığı bulunmuştur. 19 Bununla birlikte, klinik işlemlerin kriterlere sahip kontrol listeleri kullanarak puanlanması güvenilir ve pratik bir yaklaşımdır ve öğrenciler tarafından daha kolay benimsenir. 20 Klinik ortam; yeterli ve bağımsız klinik pratiği sergileyebilecek mezunların eğitimi için esastır.…”

Development of an assessment strategy in preclinical fixed prosthodontics course using virtual assessment software—Part 1

“…However, even after calibrating faculty evaluation of student work as acceptable or unacceptable, faculty members still frequently mark unacceptable student work as acceptable in practice (Haj‐Ali & Feil, ). This issue becomes even more difficult when calibrating faculty for the use of larger grading scales (Fuller, ; Haj‐Ali & Feil, ; Lilley, Bruggen Cate, Holloway, Holt, & Start, ; Salvendy, Hinton, Ferguson, & Cunningham, ; Sharaf, AbdelAziz, & El Meligy, ). Faculty assessment is also complicated by significant levels of disagreement that are often observed between graders when evaluating student performance (Fuller, ; Lilley et al, ; Salvendy et al, ).…”