Shaping Medical Assessment with Expert Consensus: A Theory-Based Summative Evaluation Blueprint for Community Medicine.

Abstract

Background Many authors have proposed summative evaluation blueprints for cognitive domain levels. But the process of calculating the overall weight was not clear, and it was done at the chapter level. This study sought to create a theory-based summative evaluation blueprint by involving experts from the All-India Institute of Medical Sciences. Methods Twenty-five experts from 10 All India Institutes of Medical Science were invited via email and contributed for summative assessment for a theory paper. Each week, the study lead team sent an online form containing 25 topics. A total of 269 topics were selected to be included in the blueprint development. An Impact and Frequency score was obtained for each topic, with a score between 1 and 3 for both its "Impact (I)" and "Frequency" (F). Results The overall response rate was 71%. "Principle of epidemiology" (14.7%) had the most weight, followed by "Epidemiology of communicable disease" (13.4%) and "National Health Program" (10%). These three chapters constituted more than one-third of the total weight of the summative assessment (theory). 20 out of 24 (83%) chapters have moderate-to-good interrater reliability. Conclusions The proposed blueprint will serve as a reference for setting the question paper and will impart objectivity, uniformity, and validity to the construction of the written assessment tool.