Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study

Wagener, Stefan; Möltner, Andreas; Timbil, Sevgi; Gornostayeva, Maryna; Jh, Schultz; Brüstle, Peter; Mohr, Daniela; A, Vander Beken; Better, Jennifer; Fries, Michael; Gottschalk, Marc; Günther, J.; Herrmann, Lisa E.; Kreisel, C; Moczko, Tobias R; Illg, Claudius; Jassowicz, Adam; Müller, Amanda; Niesert, M; Strübing, Felix; Jünger, Jana

doi:10.3205/zma000988

Cited by 7 publications

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An example of student-generated formative assessments specifically targeting competency-based progression was illustrated in a multicenter pilot study in German medical schools [ 32 ]. A core team of 17 students from the third to ninth semesters drawn from 17 universities was trained on MCQ generation and review, contributing 118 MCQs to a 144-item assessment based on a preagreed competency blueprint.…”

Section: Discussionmentioning

confidence: 99%

Assessment as Learning in Medical Education: Feasibility and Perceived Impact of Student-Generated Formative Assessments

Lakhtakia¹,

Otaki²,

Alsuwaidi³

et al. 2022

JMIR Med Educ

View full text Add to dashboard Cite

Background Self-regulated learning (SRL) is gaining widespread recognition as a vital competency that is desirable to sustain lifelong learning, especially relevant to health professions education. Contemporary educational practices emphasize this aspect of undergraduate medical education through innovative designs of teaching and learning, such as the flipped classroom and team-based learning. Assessment practices are less commonly deployed to build capacity for SRL. Assessment as learning (AaL) can be a unique way of inculcating SRL by enabling active learning habits. It charges students to create formative assessments, reinforcing student-centered in-depth learning and critical thinking. Objective This study aimed to explore, from the learners’ perspectives, the feasibility and perceived learning impact of student-generated formative assessments. Methods This study relied on a convergent mixed methods approach. An educational intervention was deployed on a cohort of 54 students in the second year of a 6-year undergraduate medical program as part of a single-course curriculum. The AaL intervention engaged students in generating assessments using peer collaboration, tutor facilitation, and feedback. The outcomes of the intervention were measured through quantitative and qualitative data on student perceptions, which were collected through an anonymized web-based survey and in-person focus groups, respectively. Quantitative survey data were analyzed using SPSS (IBM), and qualitative inputs underwent thematic analysis. Results The students’ overall score of agreement with the AaL educational intervention was 84%, which was strongly correlated with scores for ease and impact on a 5-point Likert-type scale. The themes that emerged from the qualitative analysis included prominent characteristics, immediate gains, and expected long-term benefits of engagement. The prominent characteristics included individuals’ engagement, effective interdependencies, novelty, and time requirements. The identified immediate gains highlighted increased motivation and acquisition of knowledge and skills. The expected long-term benefits included critical thinking, problem solving, and clinical reasoning. Conclusions As a form of AaL, student-generated assessments were perceived as viable, constructive, and stimulating educational exercises by the student authors. In the short term, the activity provided students with a fun and challenging opportunity to dive deeply into the content, be creative in designing questions, and improve exam-taking skills. In the long term, students expected an enhancement of critical thinking and the inculcation of student-centered attributes of self-regulated lifelong learning and peer collaboration, which are vital to the practice of medicine.

show abstract

Section: Discussionmentioning

confidence: 99%

Assessment as Learning in Medical Education: Feasibility and Perceived Impact of Student-Generated Formative Assessments

Lakhtakia¹,

Otaki²,

Alsuwaidi³

et al. 2022

JMIR Med Educ

View full text Add to dashboard Cite

show abstract

Inclusion of MCQs written by radiology residents in their annual evaluation: innovative method to enhance resident’s empowerment?

Amini

Michoux

Warnier³

et al. 2020

Insights Imaging

View full text Add to dashboard Cite

Aim: We hypothesized that multiple-choice questions written by radiology residents (MCQ resident) for their weekly case presentations during radiology staff meetings could be used along with multiple-choice questions written by radiology teachers (MCQ teacher) for their annual evaluation. The current prospective study aimed at determining the educational characteristics of MCQ resident and at comparing them with those of MCQ teacher. Methods: Fifty-one radiology residents of the first to the fifth year of training took the 2017 exam that contained 58 MCQ resident and 63 of MCQ teacher. The difficulty index, the discrimination power, and the distractor's quality were calculated in the two series of MCQs and were compared by using Student t test. Two radiologists classified each MCQ according to Bloom's taxonomy and frequencies of required skills of both MCQ series were compared. Results: The mean ± SD difficulty index of MCQ resident was statistically significantly higher than that of MCQ teacher (0.81 ± 0.1 vs 0.64 ± 0.2; p < 0.0001). The mean ± SD discrimination index of MCQ resident was statistically significantly higher than that of MCQ teacher (0.34 ± 0.2 vs 0.23 ± 0.2; p = 0.0007). The mean number of nonfunctional distractors per MCQ resident was statistically significantly higher than that per MCQ teacher (1.36 ± 0.9 vs 0.86 ± 0.9; p = 0.0031). MCQ resident required recalling skills more frequently than MCQ teacher which required more advanced skills to obtain a correct answer. Conclusions: Educational characteristics of MCQ resident differ from those of MCQ teacher. This study highlights the characteristics to optimize the writing of MCQs by radiology residents.

show abstract

Expert computer assessment of knowledge

Терентьєв

Клещов²

2018

Humanities Bulletin of Zaporizhzhe State Engineering Academy

View full text Add to dashboard Cite

The urgency of the study of "stem-education" as a factor in the development of "smartsociety" is that this kind of society (smart-society) is a continuation of information (network) and "knowledge society", which is developing on the basis of smart technologies. The concept of smart society is at the heart of modern state-owned development programs of South Korea and Japan. In South Korea, the National Social Agency has developed a "Smart Society Strategy" that introduces the technological foundations of smart societies. The central issue of Smart-society is business, which makes managing more intelligent (flexible, intelligent) and activities aimed at using knowledge and innovation. The objectives of the study are to identify the "stem-education" endpoint through the use of electronic and collective technologies , which become more massive and effective, and through the use of natural, engineering and maternal education. It is no coincidence that this concept is documented in the Europe 2020 Strategy: Smart Growth Strategy, which includes the development of an economy based on knowledge and innovation , fosters sustainable growth, more efficient use of resources, and inclusive growth and strengthening of high employment. Countries such as the United States, China, Australia, the United Kingdom, Israel, and Singapore carry out state-of-the-art stem-education programs. The methodological basis of the study is systematic, structural, axiological, synergetic methods and approaches that have allowed disclosing stem-education as the leading trend in the world of educational space. Conclusion-It is proved that stem-education is an innovation that combines the traditions of natural and mathematical education , is based on the principles of fundamentalism and knowledge-based, combining technological, organizational, material and technical resources and human capital. As a result of the development of stem-education at the expense of ICT, business processes, governance, and management reform are undergoing change, leading to economic, social and managerial processes at a higher level in some of those societies. Smart society is a new quality of society in which the combination of the use of new technologies will allow people to improve the quality of their lives. This article presents philosophical and educational reflection of smart-society as a new model of the information society and presents its impact on human (intellectual) capital. It reveals timeliness of this topic, which is innovative and hardly developed. It analyses international experience in establishment and growth of smart-society and dimensions of axiological field of smart-society, which is based on axiological matrix of information and knowledge , which are considered and being civilized dimensions of modern society. The main idea is to prove the evolution of the information society to smart-society and the possibility of establishment of smart-society in Ukraine. The analysis of smart-society formation was made and its characteristics were defined , wh...

show abstract

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study

Cited by 7 publications

References 32 publications

Assessment as Learning in Medical Education: Feasibility and Perceived Impact of Student-Generated Formative Assessments

Assessment as Learning in Medical Education: Feasibility and Perceived Impact of Student-Generated Formative Assessments

Inclusion of MCQs written by radiology residents in their annual evaluation: innovative method to enhance resident’s empowerment?

Expert computer assessment of knowledge

Contact Info

Product

Resources

About