Purpose: We have observed that students’ performance in our PreClerkship curriculum does not align well with their USMLE STEP1 scores. Students at-risk of failing or underperforming on STEP1 have often excelled in our institutional assessments. We sought to test the validity and reliability of our course-assessments in predicting STEP1 scores, and in the process generate and validate a more accurate prediction model for STEP1 performance.Methods: Student pre-matriculation and course assessment data of the Class of 2020 (n = 76) is used to generate a stepwise STEP1 prediction model, which is tested with the students of the Class of 2021 (n = 71). Predictions are generated for the end of each course in the programing language R. For the Class of 2021, predicted STEP1 score is correlated with their actual STEP1 scores and data-agreement is tested with means-difference plots. A similar model is generated and tested for the Class of 2022.Results: STEP1 predictions based pre-matriculation data are unreliable and fail to identify at-risk students (R2 = 0.02). STEP1 predictions for most year 1 courses (anatomy, biochemistry, physiology) correlate poorly with students’ actual STEP1 scores (R2 = 0.30). STEP1 predictions improve for year 2 courses (microbiology, pathology and pharmacology), but reliable predictions are based on truly integrated courses with customized NBMEs as comprehensive exams (0.66). Predictions based on these NBMEs and integrated courses are reproducible for the Class of 2022.Conclusion: MCAT and undergraduate GPA are poor predictors of students’ STEP1 scores. Partially integrated courses with biweekly assessments do not promote problem-solving skills and leave students’ at-risk of failing STEP1. Only truly integrated courses with comprehensive assessments are reliable indicators of students STEP1 preparation.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.