Impact of adopting a newly developed blueprinting method and relating it to item analysis on students’ performance

Ahmad, Rani; Hamed, Omayma

doi:10.3109/0142159x.2014.886014

Cited by 14 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The primary goal of assessment via examination is to accurately measure student achievement of desired knowledge and competencies, which are generally articulated through learning objectives. 1,2 For students, locally developed examinations convey educational concepts and topics deemed important by faculty members, which allows students to interact with those concepts and receive feedback on the extent to which they have mastered the material. 3,4 For faculty members, results provide valuable insight into how students are thinking about concepts, assist with identifying student misconceptions, and often serve as the basis for assigning course grades.…”

Section: Introductionmentioning

confidence: 99%

Best Practices Related to Examination Item Construction and Post-hoc Review

Rudolph

Daugherty

Ray

et al. 2019

American Journal of Pharmaceutical Education

View full text Add to dashboard Cite

Objective. To provide a practical guide to examination item writing, item statistics, and score adjustment for use by pharmacy and other health professions educators. Findings. Each examination item type possesses advantages and disadvantages. Whereas selected response items allow for efficient assessment of student recall and understanding of content, constructed response items appear better suited for assessment of higher levels of Bloom's taxonomy. Although clear criteria have not been established, accepted ranges for item statistics and examination reliability have been identified. Existing literature provides guidance on when instructors should consider revising or removing items from future examinations based on item statistics and review, but limited information is available on performing score adjustments. Summary. Instructors should select item types that align with the intended learning objectives to be measured on the examination. Ideally, an examination will consist of multiple item types to capitalize on the advantages and limit the effects of any disadvantages associated with a specific item format. Score adjustments should be performed judiciously and by considering all available item information. Colleges and schools should consider developing item writing and score adjustment guidelines to promote consistency.

show abstract

Section: Introductionmentioning

confidence: 99%

Best Practices Related to Examination Item Construction and Post-hoc Review

Rudolph

Daugherty

Ray

et al. 2019

American Journal of Pharmaceutical Education

View full text Add to dashboard Cite

show abstract

“…Roberts et al noted that adequate resources are not always available even at the university level to ensure the valid and reliable assessment of clinical competence in the United Kingdom [ 7 ]. Building upon earlier approaches, we proposed a methodology especially suited to programs constrained by program duration relative to a large number of objectives and seemingly endless numbers of possible test questions [ 3 , 8 ]. Based upon the authors’ experience, the 50-item multiple-choice exam is a fairly common fixture in prehospital professional development examinations.…”

Section: Discussionmentioning

confidence: 99%

“…Assessment blueprinting also offers advantages beyond its reported enhancement of content validity in examinations [ 2 ]. Ahmad et al recently reported that their blueprinting methods, which were somewhat similar to our approach, led to improved student outcomes as measured by test scores and self-reported student satisfaction measures [ 8 ]. Furthermore, in light of the potential variability in the construction of assessment blueprints, we concur with these authors and recommend that each testing entity should consider the educational complexity and duration of their programs and courses and choose a blueprinting method best suited to ensuring content validity for each instance of testing.…”

Section: Discussionmentioning

confidence: 99%

An assessment blueprint for the Advanced Medical Life Support two-day prehospital emergency medical services training program in the United States

Becker

Vassar

2015

J Educ Eval Health Prof

View full text Add to dashboard Cite

Purpose:Traditional approaches to blueprint creation may focus on fine-grained detail at the expense of important foundational concepts. The purpose of this study was to develop a method for constructing an assessment blueprint to guide the creation of a new post-test for a two-day prehospital emergency medical services training program.Methods:In order to create the blueprint, we first determined the proportions of the total classroom and home-study minutes associated with the lower- and higher-order cognitive objectives of each chapter of the textbook and the two-day classroom activities during training courses conducted from January to April 2015. These proportions were then applied to a 50-question test structure in order to calculate the number of desired questions by chapter and content type.Results:Our blueprint called for the test to contain an almost even split of lower- and higher-order cognitive questions. One-best-answer multiple choice items and extended matching-type items were written to assess lower- and higher-order cognitive content, respectively.Conclusion:We report the first known application of an assessment blueprint to a prehospital professional development education program. Our approach to blueprint creation is computationally straightforward and could be easily adopted by a group of instructors with a basic understanding of lower- and higher-order cognitive constructs. By blueprinting at the chapter level, as we have done, item-writers should be more inclined to construct questions that focus on important central themes or procedures.

show abstract

“…Other studies have also suggested that blueprinting of assessment methods not only increases their reliability and validity, but helps in student's performance result and satisfaction. (Ahmad & Hamed, 2014).…”

Section: Figure 2: Comparison Of Percentage Scores Of Pre-test and Pomentioning

confidence: 99%

‘Blueprinting in assessment’ – an online learning experience

Pawade

Mehta

Mahajan

et al. 2020

SE Asian Jnl Med Educ

View full text Add to dashboard Cite

Objective: Faculty-training on blueprinting is necessary to make medical teachers aware of threats to the attributes of assessment. The experiences and challenges of preparation and moderation of the online ML Web Session are discussed so as to facilitate the readers to conduct such activities. Methods: GSMC-FAIMER Regional Institute, Mumbai, India chose "Blueprinting in Assessment" for Web Discussion in January 2018, as a part of the GSMC-FAIMER Fellowship Programme. 32 participants were involved in an asynchronous manner, using Listserv Online Discussion Forum, from 1st to 31st January 2018. A sequential flow of the activities was aligned with chronological sequence of pre-defined objectives. Reflections were taken at the end of the month. Results: Comparison of Pre-test and Post-test analysis showed statistically significant improvement in percentage scores (p<0.05). The participants appreciated the ML Web Discussion Session activities in their reflections. Conclusion: This ML Web Session achieved the learning objectives, increased the knowledge and led to a deeper understanding of blueprinting in assessment, through comprehensive, though asynchronous, peer-group discussion.

show abstract

Impact of adopting a newly developed blueprinting method and relating it to item analysis on students’ performance

Abstract: The method adopted by using the newly developed blueprinting method and relating it to item analysis results has positive impact on the validity and reliability of students' performance results and their satisfaction in relation to intended learning outcomes.

Cited by 14 publications

References 9 publications

Best Practices Related to Examination Item Construction and Post-hoc Review

Best Practices Related to Examination Item Construction and Post-hoc Review

An assessment blueprint for the Advanced Medical Life Support two-day prehospital emergency medical services training program in the United States

‘Blueprinting in assessment’ – an online learning experience

Contact Info

Product

Resources

About