Gentle Introduction of Generalizability Theory Analysis in OSCE Using EduG for Medical Educators

Abstract

has become the gold standard of reliability analysis in OSCEs. However, little progress has been made in popularizing G-Theory. This method allows various variance calculations to determine and quantify the source of an error. EduG is the only GUI software able to calculate G-Theory. The objective of this project is to introduce the G-Theory analysis of an OSCE using EduG. Experimental Design. The assessment is a one day OSCE consisting of 10 stations per circuit. There are 3 buildings used in this exam. Each building held one complete circuit. Result. G coefficient is 0.76. The majority of error (81.4%) was due to the interaction between students (nested in location) and observations. There is no error coming from the observation itself. Mean from building A, B, and C are 57.41, 57.97, and 61.56, respectively. Conclusion. First conclusion, the possible source of error is greater in building C. Secondly, EduG is capable calculating the reliability analysis in OSCE using G-Theory.