Typical physics Ph.D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion

Abstract

This work aims to understand how effective the typical admissions criteria used in physics are at identifying students who will complete the PhD. Through a multivariate statistical analysis of a sample that includes roughly one in eight students who entered physics PhD programs from 2000-2010, we find that the traditional admissions metrics of undergraduate GPA and the Graduate Records Examination (GRE) Quantitative, Verbal, and Physics Subject Tests do not predict completion in US physics graduate programs with the efficacy often assumed by admissions committees. We find only undergraduate GPA to have a statistically significant association with physics PhD completion across all models studied. In no model did GRE Physics or GRE Verbal predict PhD completion. GRE Quantitative scores had statistically significant relationships with PhD completion in two of four models studied. However, in practice, probability of completing the PhD changed by less than 10 percentage points for students scoring in the 10 vs 90 percentile of US test takers that were physics majors. Noting the significant race, gender, and citizenship gaps in GRE scores, these findings indicate that the heavy reliance on these test scores within typical PhD admissions process is a deterrent to increasing access, diversity, and equity in physics. Misuse of GRE scores selects against already-underrepresented groups and US citizens with tools that fail to meaningfully predict PhD completion. This is a draft; see the journal for the published version.Additionally included in blue text are several responses to queries about this work.

The Role of Selective College Admissions Criteria in Interrupting or Reproducing Racial and Economic Inequities

“…Sociologists have written extensively on how standardized tests (in both their form and content) favor the already privileged (e.g., Miller et al, 2019;Posselt et al, 2012;Soares, 2010), yet most selective colleges continue to use these tests as a way to distinguish between applicants. Increasingly, many colleges consider a variety of criteria alongside test scores in what has been publicized as an effort to expand racial and economic diversity through holistic review and/or test-optional policies.…”

Using machine learning to understand physics graduate school admissions

“…While we did not have access to other criterion included in the Potvin et al such as application essays, research experiences, and recommendation letters, we were still able to create a model that correctly predicted whether an applicant would be admitted with 75% accuracy based solely off the applicant's undergraduate GPA and physics GRE score. While undergraduate GPA is a significant predictor of completing a physics PhD, physics GRE is not, as those scoring near the top of the physics GRE only have a 7% higher probability of completing their PhD than those scoring near the bottom [27]. As the GRE is not associated with completing a doctoral degree and is known to favor persons from majority groups in science [28], the outsized role of the GRE in the admissions process should be questioned.…”

A framework for improving diversity work in physics

“…Previous research indicates that the Graduate Record Exam is a barrier to participation in graduate education [33,[51][52][53]. The statement of AAPT recommending that the GRE be eliminated from the admissions process is an effort to remove the barrier [54].…”