Using Generalizability Theory to Assess the Score Reliability of Communication Skills of Dentistry Students

Abstract

The goal of this study is to determine the reliability of the performance points of dentistry students regarding communication skills and to examine the scoring reliability by generalizability theory in balanced random and fixed facet (mixed design) data, considering also the interactions of student, rater and duty. The study group of the research consists of the 16 dentistry students who took communication skills elective course in Hacettepe University in 2014-2015 academic year. In the study, the variability sources are taken as student (s), rater (r), occupation (o) and task (t). 16 students are assigned with 16 different duties and whether the students fulfill the duties is rated by 8 raters from two different occupation groups, 4 standard patients and 4 academicians. On these facets using different designs G study has been performed. Within the scope of this study the sxrxt design where all variability sources are crossed; sx(o:r)xt design where the occupation nested on rater as the result of scoring of the students by two different occupation groups as standard patient and academicians; sxaxt and sxspxt facets where rater facet is taken as fixed facet according to occupation groups (sp: Standard Patient, a: Academician) were examined. For each design estimated variance components are discussed separately.

Variance components of ratings of physician-patient communication: A generalizability theory analysis

“…The findings of this study are consistent with previous studies applying g-theory to examine ratings from neutral observers [ 24 , 25 ] or in other contexts [ 14 , 41 ] and are emphasized by studies investigating PPC ratings using other methods [ 18 , 42 ]. Two studies examined ratings of PCC assessed from neutral observers to evaluate students communication competence in the context of health professional training programs [ 24 , 25 ]. It was shown that students’ communication competence could only be explained to a small extent (3% [ 24 ]; 11% [ 25 ]).…”

“…Two studies examined ratings of PCC assessed from neutral observers to evaluate students communication competence in the context of health professional training programs [ 24 , 25 ]. It was shown that students’ communication competence could only be explained to a small extent (3% [ 24 ]; 11% [ 25 ]). This was also shown by a study examining the cultural competency of attending physicians using g-theory [ 41 ].…”

“…This process allows decision makers to enhance their understanding of the information included in the ratings and to draw more valid conclusions concerning the generalizability of PPC ratings across different measurement conditions. Two previous studies applying g-theory in the context of PPC investigated students’ communication competence in the field of health care professional training programs [ 24 , 25 ]. Both studies investigated PPC ratings assessed from observers.…”

Using Generalizability Theory to Investigate the Reliability of Scores Assigned to Students in English Language Examination in Nigeria

“…Therefore, a certain number of items would be needed to obtain dependable scores. Uzun et al (2018) and de Vries (2012) also pointed out that increasing the number of raters or occasions would increase the score dependability when rater and occasion are considered as facets. Li, Shavelson, Yin and Wiley (2015)…”

Students' Mathematical Communication Skills in Solving Quadratic Equation Problems

“…Mathematical communication skills are one of the competencies that must be owned by students today. Mathematical communication skills are defined as the ability to reflect an image into mathematical ideas (Nuraida & Amam, 2019;Uzun et al, 2018). Mathematical problems by using symbols and providing explanations in their own language by writing them mathematically (Al-Momany, 2015;Rahmi et al, 2017).…”