Testing pays off twice: Potentials of practice tests and feedback regarding exam performance and judgment accuracy

Naujoks, Nick; Harder, Bettina; Händel, Marion

doi:10.1007/s11409-022-09295-x

Cited by 12 publications

(9 citation statements)

References 70 publications

(123 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, our results might suggest that providing only corrective feedback is not appropriate in formative assessment activities with mobile‐based assessment systems. However, recent research seems to demonstrate that the provision of feedback during practice tests is effective for learning (Naujoks et al, 2022). Nevertheless, research on assessment feedback is still in its infancy so further research is needed to identify how assessment feedback in the context of mobile formative assessment activities might have a positive impact on students' learning performance (Bikanga, 2018).…”

Section: Discussionmentioning

confidence: 99%

Determinants of student performance with mobile‐based assessment systems for English as a foreign language courses

Bacca‐Acosta

Fabregat

Baldiris

et al. 2023

Computer Assisted Learning

View full text Add to dashboard Cite

Background Mobile‐based assessment has been an active area of research in the field of mobile learning. Prior research has demonstrated that mobile‐based assessment systems positively affect student performance. However, it is still unclear why and how these systems positively affect student performance. Objectives This study aims to identify the determinants of student performance during students' use of a mobile‐based assessment application in a formative assessment activity as part of English as a Foreign Language (EFL) courses in higher education. Methods A structural model based on hypotheses was validated using PLS‐SEM with data from the interaction of 127 students of eight EFL courses from the A1 and A2 levels of English that used a mobile‐based assessment system for a period of 4 weeks. Automatic data collection in the application and self‐reported instruments were applied. Results and Conclusions Use of scaffolding mechanisms, time on‐task and reported effort are strong predictors of students' learning outcomes. The use of scaffolding strategies predicts students' time on‐task. The provision of corrective feedback is not a predictor of students' learning performance but predicts other constructs such as perceived usefulness and the behavioural intention to use. Implications Mobile‐based assessment systems should include scaffolding mechanisms and integrate strategies to increase the perceived relevance of the formative assessment activity to increase the student learning performance. Scaffolding mechanisms are also useful to increase the student time on‐task in the formative assessment activity. In mobile‐based formative assessment activities more elaborated forms of feedback other than corrective feedback are needed to increase student performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Determinants of student performance with mobile‐based assessment systems for English as a foreign language courses

Bacca‐Acosta

Fabregat

Baldiris

et al. 2023

Computer Assisted Learning

View full text Add to dashboard Cite

show abstract

“…However, the benefits of practice questions are highly dependent on how they are incorporated into a student's overall study plan. The completion of practice questions allows students to metacognitively monitor and evaluate their current level of knowledge, including any gaps in knowledge (Naujoks et al, 2022). Each subsequent set of practice questions completed improves the accuracy of the students' judgments of their knowledge, providing them with a better idea of their examination preparedness (Naujoks et al, 2022).…”

Section: Discussionmentioning

confidence: 99%

“…Both repeating and top‐performing students also relied heavily on practice questions to prepare for the first examination, but the difference in how they used these practice questions suggests a discrepancy in their metacognitive ability. Practice questions are well‐documented as a popular and effective study strategy in the medical education literature (Deng et al, 2015; Nayak & Erinjeri, 2008; Roediger & Butler, 2011; McNulty et al, 2012; Mains et al, 2015; Naujoks et al, 2022; Wenger et al, 2009). However, the benefits of practice questions are highly dependent on how they are incorporated into a student's overall study plan.…”

Section: Discussionmentioning

confidence: 99%

Pre‐ and post‐examination reflections of first‐year medical students in an integrated medical anatomy course

Cale,

Hoffman,

McNulty

2023

Anatomical Sciences Ed

View full text Add to dashboard Cite

Due to the rigor and pace of undergraduate medical anatomy courses, it is not uncommon for students to struggle and fail initially. However, repetition of coursework places an additional burden on the student, instructor, and institution. The purpose of this study was to compare the exam preparation strategies of repeating and non‐repeating students to identify areas where struggling students can be supported prior to course failure. As part of their integrated anatomy course, first‐year medical students at Indiana University completed a metacognitive Practice‐Based Learning and Improvement (PBLI) assignment prior to and after their first exam. In the PBLIs, students were asked to reflect on their exam preparation strategies, confidence, and satisfaction, as well as their predicted and actual exam performance. PBLI responses from non‐repeating and repeating students were then analyzed quantitatively and qualitatively. A total of 1802 medical students were included in this study, including 1751 non‐repeating and 51 repeating students. Based on their PBLI responses, non‐repeating students were appropriately confident, somewhat satisfied, and more accurate when predicting their exam performance. Repeating students were overconfident, dissatisfied, and inaccurate when predicting their first exam performance on their initial, unsuccessful attempt but were more successful on their second, repeat attempt. Qualitative analysis revealed that repeating students aimed to improve their studying by modifying their existing study strategies and managing their time more effectively. In conjunction with other known risk factors, these insights into repeater and non‐repeater exam preparation practices can help anatomy educators better identify and support potential struggling students.

show abstract

“…This lack of benefit is contrary to current theory and what has been found in prior studies on practice tests, which show that repeated study and regular testing result in better organization of knowledge and improve application of knowledge to new contexts. [7][8][9]12,13 Although we have no way of defining causation for this correlation, one explanation might be that the questions in the SAEM practice tests were not accepted for inclusion in the EM-M4 examination. Reasons for exclusion included poor item discrimination values, questions not addressing the National EM M4 curriculum, and item-writing flaws.…”

Section: Story Et Al Saem Practice Tests Do Not Improve National Em C...mentioning

confidence: 99%

“…Testing can have multiple learner benefits with respect to memory retrieval and long-term retention. [7][8][9] The format of the tests requires asking the right questions in the right format. 9 As the SAEM practice tests were made of questions not selected for the EM-M4 exam there may be a concern as to the quality of the questions.…”

Section: How Does This Improve Population Health?mentioning

confidence: 99%

Taking More Society for Academic Emergency Medicine Practice Tests Does Not Lead to Improved National EM-M4 Exam Scores

Story

Guo

Vallevand

et al. 2023

WestJEM

View full text Add to dashboard Cite

Introduction: Emergency medicine (EM) is a required clerkship for third-year medical students, and an elective EM acting internship (AI) is available to fourth-year students at our institution. The Society for Academic Emergency Medicine’s (SAEM) National Emergency Medicine M4 Examination (EM-M4) is administered to students at the end of the EM AI experience. To prepare for the exam, students gain access to 23 practice tests available from SAEM. In this study we investigate the correlation between the number of practice tests taken and EM-M4 performance. Methods: We collected data for EM-M4 and the US Medical Licensing Exam (USMLE) Step 2 Clinical Knowledge (CK) from students completing a MS4 EM clerkship in consecutive medical school classes from 2014-2017 at a private medical school. In addition, we collected data during the clerkship on the number of practice exams taken and whether a comprehensive practice exam was taken. We analyzed the study population three ways to determine whether the number of practice tests impacted final exam results: a binary distribution (1-11 or 12-23 tests taken); quaternary distribution (1-6, 7-12, 13-18, or 19-23 tests taken); and individual test variability (1,2,3,…22,23 tests taken). Complete data for 147 students was used for data analysis. Results: The EM-M4 showed moderate (r = 0.49) correlations with USMLE Step 2 CK. There was no significant difference in EM-M4 performance in the binary analysis (P ≤ 0.09), the quaternary analysis (P ≤ 0.09), or the continuous variable analysis (P ≤ 0.52). Inclusion of a comprehensive practice test also did not correlate with EM-M4 performance (P ≤ 0.78). Conclusion: Degree of utilization of SAEM practice tests did not seem to correlate with performance on the EM-M4 examination at our institution. This could be due to many factors including that the question bank is composed of items that had poor item discrimination, possible inadequate coverage of EM curriculum, and/or use of alternative study methods. While further investigation is needed, if our conclusions prove generalizable, then using the SAEM practice tests is an extraneous cognitive load from a modality without proven benefit.

show abstract

Testing pays off twice: Potentials of practice tests and feedback regarding exam performance and judgment accuracy

Cited by 12 publications

References 70 publications

Determinants of student performance with mobile‐based assessment systems for English as a foreign language courses

Determinants of student performance with mobile‐based assessment systems for English as a foreign language courses

Pre‐ and post‐examination reflections of first‐year medical students in an integrated medical anatomy course

Taking More Society for Academic Emergency Medicine Practice Tests Does Not Lead to Improved National EM-M4 Exam Scores

Contact Info

Product

Resources

About