A Study of the Validity of the New MCAT Exam

Abstract

Purpose To conduct a study of the validity of the new Medical College Admission Test (MCAT). Method Deidentified data for first- and second-year medical students (185 women, 54.3%; 156 men, 45.7%) who matriculated in 2016 and 2017 to the University of Minnesota Medical School–Twin Cities were included. Of those students, 220 (64.5%) had taken the new MCAT exam and 182 (53.4%) had taken the old MCAT exam (61 [17.9%] had taken both). The authors calculated descriptive statistics and Pearson product moment correlations (r) between new and old MCAT section scores. They conducteda regression analysis of MCAT section scores with Step 1 scores and with preclerkship course performance. They also conducted an exploratory factor analysis (principal component analysis with varimax rotation) of MCAT scores, undergraduate grade point average, Step 1 scores, and course performance. Results The new MCAT exam section mean score percentiles ranged from 72 to 78 (mean composite score percentile of 80). The old MCAT exam section mean score percentiles ranged from 84 to 88 (mean composite score percentile of 83). The pattern of correlations among and between new and old MCAT exam section scores (range of r: 0.03–0.67; P < .01) provided evidence of both divergent and convergent validities. Backward multiple regression of new MCAT exam section scores and Step 1 scores resulted in a multiple R of .440; the same analysis with Human Behavior course performance as the dependent variable provided a similar solution with the expected sections of the new MCAT exam (multiple R = .502). The factor analysis resulted in 4 cohesive, theoretically meaningful factors: biomedical knowledge, basic science concepts, cognitive reasoning, and general achievement. Conclusions This study provided empirical evidence of multiple types of validity for the new MCAT exam.

A multi‐pronged, antiracist approach to optimize equity in medical school admissions

“…Both in and outside of the medical career space, performance on one standardised exam is seen as a predictor of performance on the next, and thus an applicant's MCAT performance is judged as a predictor of USMLE success. Both the previous and ongoing MCAT validity studies have suggested that the combined GPA and MCAT credentials are more predictive than either factor alone 22,23 . Preliminary data in the 2015 MCAT Validity Study support the notion that 2015 MCAT scores correlate with medical student advancement and Step I performance, however, an internal review at our institution did not corroborate this with statistical significance, and only a weak correlation was observed.…”

Relationships between preadmission variables and academic outcomes for postbaccalaureate students in medical school

“…Our results are in the expected pattern of magnitude that previous studies have demonstrated (Ferguson et al, 2002 ; Gauer et al, 2016 ; Kulatunga-Moruzi & Norman, 2002 ; Song et al, 2011 ). For example, in a recent study, the four new MCAT exam section scores explained 19% of the variance on USMLE Step 1 performance (Violato et al, 2020 ). Adding the PBPM program GPA explained an additional 10% and 7% for preclerkship and USMLE Step 1 performance, respectively.…”

The influence of machine learning-based knowledge management model on enterprise organizational capability innovation and industrial development

“…The entire questionnaire survey process was conducted with the consent of the participants; the questionnaire was not open to the public and only for research purposes. In order to test the reliability of the questionnaire, the Cronbach's α coefficient is adopted for evaluation [31], and the validity of the questionnaire is analyzed by principal component analysis (PCA) [32].…”