Examining the predictive validity of the Duolingo English Test: Evidence from a major UK university

Abstract

The COVID-19 pandemic has changed the university admissions and proficiency testing landscape. One change has been the meteoric rise in use of the fully automated Duolingo English Test (DET) for university entrance purposes, offering test-takers a cheaper, shorter, accessible alternative. This rapid response study is the first to investigate the predictive value of DET test scores in relation to university students’ academic attainment, taking into account students’ degree level, academic discipline, and nationality. We also compared DET test-takers’ academic performance with that of students admitted using traditional proficiency tests. Credit-weighted first-year academic grades of 1881 DET test-takers (1389 postgraduate, 492 undergraduate) enrolled at a large, research-intensive London university in Autumn 2020 were positively associated with DET Overall scores for postgraduate students (adj. r = .195) but not undergraduate students (adj. r = −.112). This result was mirrored in correlational patterns for students admitted through IELTS ( n = 2651) and TOEFL iBT ( n = 436), contributing to criterion-related validity evidence. Students admitted with DET enjoyed lower academic success than the IELTS and TOEFL iBT test-takers, although sample characteristics may have shaped this finding. We discuss implications for establishing cut scores and harnessing test-takers’ academic language development through pre-sessional and in-sessional support.

Speaking performances, stakeholder perceptions, and test scores: Extrapolating from the Duolingo English test to the university

“…When examining subgroups, however, relationships proved slightly stronger for some majors (business, science, engineering) and notably stronger for some nationalities within majors (e.g., Chinese business majors). Isaacs et al (2023) reported similar patterns among relationships between GPA and several English tests used for admission purposes, including the TOEFL iBT, IELTS, and DET. These findings highlight some of the nuances in linking high stakes assessment to other formal evaluations of academic success, including heterogeneity among student subgroups and the degree to which spoken language ability can be expected to account for particular outcomes.…”

“…These correlations are similar to those between IELTS and TOEFL iBT, with IELTS and TOEFL iBT total scores correlated at r = .73 and speaking scores at r = .57 (Educational Testing Service, 2010). As a relatively new test, however, the DET is only beginning to accumulate a body of research evidence to support score use in academic settings beyond correlations with other tests used for similar purposes (but see Isaacs et al, 2023).…”

The development of a Chinese vocabulary proficiency test (CVPT) for learners of Chinese as a second/foreign language

“…Although the CVPT significantly predicted participants' cloze test scores, the extrapolation inference or the conventional concurrent or predictive validity of the CVPT is recommended for future research (e.g., Isaacs et al, 2023), such as how the observed CVPT scores link test-takers' performance in other Chinese vocabulary breadth tests, or vocabulary translation behaviours in reading books, and other relevant tasks beyond the test.…”