Comparison of the validity of bookmark and Angoff standard setting methods in medical performance tests

Abstract

Background One of the main processes of determining the ability level at which a student should pass an assessment is standard setting. The current study aimed to compare the validity of Angoff and bookmark methods in standard-setting. Method 190 individuals with an M.Sc. degree in laboratory science participated in the study. A test with 32 items, designed by a group of experts, was used to assess the laboratory skills of the participants. Moreover, two groups each containing 12 content specialists in laboratory sciences, voluntarily participated in the application of the Angoff and bookmark methods. To assess the process validity, a 5-item questionnaire was asked from two groups of panelists. To investigate the internal validity, the classification agreement was calculated using the kappa and Fleiss’s Kappa coefficient. External validity was assessed by using five indices (correlation with criterion score, specificity, sensitivity, and positive and negative predictive values of correlation test with criterion score). Results The results showed that the obtained cut-scores was 17.67 for Angoff and 18.8 for bookmark. The average total of items related to the quality of the execution process was 4.25 for the Angoff group and 4.79 for the bookmark group. Pass rates pass rates percentages for the Angoff and bookmark group were 55.78 and 41.36, respectively. Correlations of passing/failing, between employer ratings and test scores were 0.69 and 0.88 for Angoff and bookmark methods, respectively. Conclusion Based on the results, it can be concluded that the process and internal validities of the bookmark method were higher than the Angoff method. For evaluation of the external validity (concordance of the cut score with the criterion score), all five external validity indices supported the bookmark method.

Psychometric Analysis of the Czech Version of the Toronto Empathy Questionnaire

“…An association between higher empathy scores and female gender is in line with the results of studies utilizing the TEQ to detect gender differences in empathy [ 13 , 16 , 17 , 63 , 65 ]. Higher empathy in females can be explained by several mechanisms, such as social expectations (i.e., society, in general, expects females to be more empathetic) [ 71 ], hormonal factors (i.e., different neuroendocrine functioning) [ 72 ] or possibly differences in brain structure [ 73 ].…”

“…This unidimensional measure yields a high internal consistency (Cronbach’s alpha > 0.85) and a high test-retest reliability ( r = 0.87) [ 2 ]. Its validity was supported via positive correlation with the Empathy Quotient [ 1 ] ( r = 0.80), the Reading the Mind in the Eyes test [ 17 ] (r = 0.35) and via negative correlation with the Autism Quotient [ 18 ] ( r = −0.30) [ 2 ]. To date, the TEQ as validated also in Turkey [ 19 ], China [ 3 ] or Korea [ 20 ].…”

Impact of COVID-19 Pandemic on Academic Activity and Health Status among Romanian Medical Dentistry Students; A Cross-Sectional Study

“…In light of national COVID-19 infection-control recommendations, dental faculties had to suspend in-person instruction, and teaching activity was conducted online. Some researchers who focused on online dental education pointed out the rapid change adopted by dental faculties in many countries, such as the USA [ 6 , 7 , 8 , 9 ], Germany [ 10 ], Italy [ 11 ], China [ 12 , 13 ], Chile [ 14 ], Brazil [ 8 , 15 , 16 ], Pakistan [ 17 ], Nepal [ 18 , 19 ], Indonesia [ 20 ], India [ 21 , 22 ], Jordan [ 23 ], Romania [ 24 , 25 , 26 ], Portugal [ 27 , 28 ], Cyprus [ 29 ], Australia [ 8 , 30 ], and Spain [ 31 ]. During the first year of the pandemic, the academic research teams additionally wanted to identify the differences between countries in what aspects of dental education were disrupted.…”

A preliminary study on applying holistic health care model on medical education behavioral intention: a theoretical perspective of planned behavior

“…As a result, students have different behavioral intentions for holistic medical education. Ottenhoff-de Jonge et al [ 30 ] showed that the educational beliefs of medical educators influence their teaching practices. It is possible that, if the hospital can appropriately adjust and improve the workload of clinical teachers, it will help the implementation of holistic medical education and potentially help the development of clinical teachers’ work.…”