Question vetting: theory and practice

Wadi, Majed Mohammed

doi:10.5959/eimj.v4i1.29

Cited by 11 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Validity, as applied to test scores, refers to the accuracy with which the scores measure a particular ability of interest [21]. Flawed questions can negatively affect test validity [6,9,22]. For example, they can be more difficult than questions that adhere to standard guidelines.…”

Section: Discussionmentioning

confidence: 99%

“…Studies have revealed, however, that examinations from medical school through CME are rife with questions that violate accepted standard question-writing principles [1][2][3][4][5][6]. In the medical education literature, the term for these nonstandard questions is "flawed" [1][2][3][7][8][9]. Flawed items can be as much as 15 percentage points more difficult to answer correctly than questions that adhere to proper guidelines [3], and tests that include them have a failure rate elevated as much as 25% [2].…”

mentioning

confidence: 99%

See 1 more Smart Citation

JOURNAL CLUB: Prevalence of Flawed Multiple-Choice Questions in Continuing Medical Education Activities of Major Radiology Journals

DiSantis

Ayoob²,

Williams³

2015

American Journal of Roentgenology

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

JOURNAL CLUB: Prevalence of Flawed Multiple-Choice Questions in Continuing Medical Education Activities of Major Radiology Journals

DiSantis

Ayoob²,

Williams³

2015

American Journal of Roentgenology

View full text Add to dashboard Cite

show abstract

“…Their findings confirmed the correlation of all the modalities. On the other hand, students' scores resulting from different assessment modalities may not be correlated as shown in other studies [16]. In some other studies, the overall reliability of correlating scores of different assessment modalities had been questioned.…”

Section: Discussionmentioning

confidence: 87%

Relation of the final year performance in different assessment modalities in Final MBBS Pediatrics, College of Medicine, Bahri Universit

Idris¹,

Rahma²,

Elfaki³

et al. 2017

J Contemp Med Edu

View full text Add to dashboard Cite

Background: Many tools and instruments can be used for the assessment of medical students including multiple choice questions (MCQs), modified essay questions (MEQs), objective structured practical examination (OSPE), problems, short answer questions (SAQs), extended matching questions (EMQs), and objective structured clinical examination (OSCE). The golden rule in assessment is to use multiple methods of assessment to provide students adequate opportunities to perform well. Using variable assessment tools will allow for making use of their advantages and reduce their disadvantages as much as possible. For correlation of combinations of assessment tools, alignment between objectives and instruction should be made. Many studies demonstrated different correlation with different forms of assessment, namely, MCQs, EMQs, MEQs, OSPE, OSCE, and SAQs. The objectives of this study were to determine the correlation of the students' scores in the four assessment modalities used for the final students in pediatrics and to determine the correlation of the combined score of written assessment and the combined score of practical assessment with the composite score. Methodology: This was a descriptive, cross-sectional study. A correlational design was used. The scores of 219 students in four methods of assessment adopted in pediatrics were analyzed. The assessment methods were MCQs, problems, OSPE, and OSCE. SPSS version 21 was used to compute the Pearson's correlation for each of the four methods with each other and with the composite scores. Results: The Pearson's r for the combined score of MCQs and problems with the composite score of all four assessment was 0.966 while 0.971 was the Pearson's r for the combined score for OSCE and OSPE with the composite score. Pearson's r of each of the assessment methods with composite scores were determined. These were 0.924, 0.901, 0.953, and 0.824 for MCQs, problems, OSCE, and OSPE, respectively. The correlation of each of the methods of assessment with each other revealed a Pearson's r correlation ranging from 0.977 to 0.819. All the correlations were highly significant. Conclusion: Analysis of the students' scores in the four assessment methods adopted in dermatology showed highly significant correlation. Scores in the written and practical assessment were highly correlated. The curriculum type adopted in the college was the SPICES model. The rationale for this study was to ensure that the assessment adopted for this course is sounded and could result in good quality of graduates. The objectives of this study were first to determine the correlation of the students' scores in the four assessment modalities used for the final students in pediatrics. Second, to determine the correlation of the combined score of written assessment and the combined score of practical assessment with the composite score. METHODOLOGYThis was a descriptive, cross-sectional, correlation study conducted at the faculty of medicine, BU in the period JuneOctober 2016. Ethical clearance was obtained from t...

show abstract

“…From this viewpoint, vetting of assessment tools is a crucial process. Question vetting is the process of reviewing and evaluating question items according to specified criteria with the intention to detect flaws and to edit them accordingly to improve their quality [3]. This step is important to sustain the validity of test items [questions] and avoid or minimize the threats.…”

mentioning

confidence: 99%

“…So, once an item is constructed, it should undergo a critical review by a review (or vetting) committee. The aim is to treat test items appropriately by removing flaws and making them as clear and as understandable as possible [3].…”

mentioning

confidence: 99%

Question Vetting: The Process to Ensure Quality in Assessment of Medical Students

Gopalakrishnan

Udayshankar

2014

JCDR

View full text Add to dashboard Cite

Over the past decade, medical schools have made new efforts to provide accurate, reliable, and timely assessments of the competence of trainees. All methods of assessment have strengths and intrinsic flaws. Written examination questions forms an important item in knowledge assessment and are typically classified according to whether they are open-ended or multiple choices. At present questions are prepared casually just before the examination and are not put through any quality check to correct any possible mistakes or ambiguity in the questions. This may lead to confusion or wrong understanding of the questions by the students which will be reflected in their answers as well. Question Vetting is the process when an expert person examines or evaluates and edits questions to make it free of any mistakes. The importance of vetting and re-vetting of examination questions cannot be overstated in the present scenario of medical education. There is a prescribed structure and protocol of question setting and vetting for examinations in medical education programmes. The areas that need to be examined during question vetting are for technical accuracy, content level and language aspects. The use of the prescribed structure and protocol of question vetting ensures a consistently high standard of question presentation during examinations which will help the students to understand the questions better and to answer them correctly. This article is intended to highlight the importance of introducing question vetting in our medical education scenario in order to improve the quality of assessment of medical students.

show abstract

Question vetting: theory and practice

Cited by 11 publications

References 21 publications

JOURNAL CLUB: Prevalence of Flawed Multiple-Choice Questions in Continuing Medical Education Activities of Major Radiology Journals

JOURNAL CLUB: Prevalence of Flawed Multiple-Choice Questions in Continuing Medical Education Activities of Major Radiology Journals

Relation of the final year performance in different assessment modalities in Final MBBS Pediatrics, College of Medicine, Bahri Universit

Question Vetting: The Process to Ensure Quality in Assessment of Medical Students

Contact Info

Product

Resources

About