Weighting checklist items and station components on a large-scale OSCE: Is it worth the effort?

Sandilands, Debra; Gotzmann, Andrea; Roy, Marguerite; Zumbo, Bruno D.; Champlain, André De

doi:10.3109/0142159x.2014.899687

Cited by 11 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Scoring also depends on the selection of response options (e.g. dichotomous checklist versus global rating scale, the number and weighting of response options, and the choice of specific scale anchors), scoring rubric and procedures, and item analysis. Fairness requires consideration of whether everyone was given a similar test, the resolution of which may include standardisation and approaches to enhance test security.…”

Section: Kane's Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

A contemporary approach to validity arguments: a practical guide to Kane's framework

et al. 2015

View full text Add to dashboard Cite

show abstract

Section: Kane's Frameworkmentioning

confidence: 99%

“…Scoring also depends on the selection of response options (e.g. dichotomous checklist versus global rating scale, 35 the number 36 and weighting of response options, 37 and the choice of specific scale anchors),…”

Section: Scoring Inferencementioning

confidence: 99%

A contemporary approach to validity arguments: a practical guide to Kane's framework

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Such weights are usually determined via expert judgement of item-writers (Kahraman et al 2008;American Educational Research Association 2014, p. 93). There is a considerable body of literature, both internal and external to medical education (Wainer 1976;Streiner & Norman 2008;Sandilands et al 2014) that argues that such differential item weighting schemes make little difference to measures of overall assessment quality, and to pass/fail rates at overall cohort levels. Hence, these studies generally contend that, rather than developing 'complex' (Sandilands et al 2014) weighting schemes, assessment writers would be better off spending their time and effort on other aspects of test development.…”

Section: To Weight or Not To Weight?mentioning

confidence: 99%

Shining a spotlight on scoring in the OSCE: Checklists and item weighting

et al. 2020

View full text Add to dashboard Cite

IntroductionThere has been a long running debate about the validity of item-based checklist scoring of performance assessments like OSCEs. In recent years, the conception of a checklist has developed from its dichotomous inception into a more 'key-features' and/or chunked approach, where 'items' have the potential to become weighted differently, but the literature does not always reflect these broader conceptions. MethodsWe consider theoretical, design and (clinically trained) assessor issues related to differential item weighting in checklist scoring of OSCEs stations. Using empirical evidence, this work also compares candidate decisions and psychometric quality of different item-weighting approaches (i.e. a simple 'unweighted' scheme versus a differentially weighted one). ResultsThe impact of different weighting schemes affect approximately 30% of the key borderline group of candidates, and 3% of candidates overall. We also find that measures of overall assessment quality are a little better under the differentially weighted scoring system. Discussion and conclusionDifferentially weighted modern checklists can contribute to valid assessment outcomes, and bring a range of additional benefits to the assessment. Judgment about weighting of particular items should be considered a key design consideration during station development, and must align to clinical assessor expectations of the relative importance of sub-tasks.

show abstract

“…Isi checklist tidak semata disusun berdasarkan segi praktis tetapi juga sisi ilmiah menyertakan evidenced-based medicine. 10 Checklist sebagai alat penilai juga perlu diuji validitas dan reliabilitasnya 11,12 sehingga dapat menjadi alat ukur yang diakui sebagai standar seperti Global Rating Scale dan Procedure-Based Assessment yang sudah diakui sebagai standar kelas satu Accreditation Council for Graduate Medical Education (ACGME). 5,13 Oleh karenanya, rekam jejak desain dan penggunaan checklist perlu dibukukan.…”

Section: Pembahasanunclassified

Memaknai Checklist bagi Mahasiswa Kedokteran dan Profesi Kesehatan: Resensi Buku

Carolyn¹,

Yulianto²,

Rukmini³

2015

JPKI

View full text Add to dashboard Cite

Atul Gawande dalam bukunya menceritakan tentang berbagai peristiwa tidak terduga yang terjadi dalam rutinitas karya bidang kedokteran. Rutinitas yang dilakukan karena kecerobohan atau ketidak telitian seseorang dapat berdampak pada orang lain, dalam hal ini pasien yang hidupnya bergantung pada tiap keputusan dan tindakan dokter. Dimulai dari pengalamannya sebagai dokter bedah, Atul Gawande menceritakan kompleksnya manajemen yang harus dilakukan seorang dokter terhadap pasiennya dan mudahnya seorang dokter dapat tertipu oleh gejala pasien. Dalam bidang medis penanganan yang cepat dan tepat diperlukan pasien-pasien UGD maupun bedah, kesalahan yang kecil dalam tindakan medis atau keluputan seorang dokter melakukan anamnesis dapat mengakibatkan kelumpuhan bahkan kematian pasien.

show abstract

Weighting checklist items and station components on a large-scale OSCE: Is it worth the effort?

Cited by 11 publications

References 14 publications

A contemporary approach to validity arguments: a practical guide to Kane's framework

A contemporary approach to validity arguments: a practical guide to Kane's framework

Shining a spotlight on scoring in the OSCE: Checklists and item weighting

Memaknai Checklist bagi Mahasiswa Kedokteran dan Profesi Kesehatan: Resensi Buku

Contact Info

Product

Resources

About