Testing the validity of value‐added measures of educational progress with genetic data

Abstract

Value‐added measures of educational progress have been used by education researchers and policy‐makers to assess the performance of teachers and schools, contributing to performance‐related pay and position in school league tables. They are designed to control for all underlying differences between pupils and should therefore provide unbiased measures of school and teacher influence on pupil progress, however, their effectiveness has been questioned. We exploit genetic data from a UK birth cohort to investigate how successfully value‐added measures control for genetic differences between pupils. We use raw value‐added, contextual value‐added (which additionally controls for background characteristics) and teacher‐reported value‐added measures built from data at ages 11, 14 and 16. Sample sizes for analyses range from 4,600 to 6,518. Our findings demonstrate that genetic differences between pupils explain little variation in raw value‐added measures but explain up to 20% of the variation in contextual value‐added measures (95% CI = 6.06% to 35.71%). Value‐added measures built from teacher‐rated ability have a greater proportion of variance explained by genetic differences between pupils, with 36.3% of their cross‐sectional variation being statistically accounted for by genetics (95% CI = 22.8% to 49.8%). By contrast, a far greater proportion of variance is explained by genetic differences for raw test scores at each age of at least 47.3% (95% CI: 35.9 to 58.7). These findings provide evidence that value‐added measures of educational progress can be influenced by genetic differences between pupils, and therefore may provide a biased measure of school and teacher performance. We include a glossary of genetic terms for educational researchers interested in the use of genetic data in educational research.

Can education be personalised using pupils’ genetic data?

“…There was a large amount of overlap in the polygenic score distribution between pupils in the top 10% of attainers and all others; while pupils with a high polygenic score are more likely to be high attainers, genetics do not determine high attainment. High academic attainment is due to both environmental and genetic factors, including social background 19 , teacher bias 22,23 , the home and school environment 24,25 , and luck. It is also possible that the quality of family and school environments may constrain or support pupils' ability to exploit their genetic propensity to education.…”

“…The usefulness of polygenic scores for educational research has been previously demonstrated for example in assessing the effectiveness of teachers and schools 12,23 ; selection differences between schools 28,29 ; and social mobility over time and space 30 . However, our results demonstrate that while polygenic scores are useful for investigating group differences, they do not provide suitable value for routine use by teachers and schools above phenotypic data to predict a pupil's attainment.…”

Why are education, socioeconomic position and intelligence genetically correlated?

“…However, the strength of this bias may be smaller for education test scores which are likely to capture a more cognitive aspect of educational performance than the more social aspect of education that years of education captures. As we have discussed in more detail elsewhere, 61 the high heritabilities we observed may also reflect genuine differences due to the spatiotemporal homogeneity of the ALSPAC cohort. The mechanisms we investigated may also have larger effects in the ALSPAC study as a regional cohort than in other data samples; the impact of these mechanisms on more geographically dispersed studies such as UK Biobank is currently unknown.…”

Genetic Associations with Mathematics Tracking and Persistence in Secondary School

“…Genetic information about students is as predictive of success in schooling as family income 8 , and genetic differences between students could similarly confound measures of school quality 16,24 . Indeed, one previous study in the U.K. found that value-added measures of teacher quality were correlated with the average education-PGS of students, suggesting that conventional models of educational quality that fail to consider genetic differences between students might lead to biased conclusions, 44 whereas incorporating data on student genetics might help clarify the impact of schools: In this study, we examined only one school-level characteristic (proportion of students whose mothers graduated high school), so much work remains to be done to identify the characteristics of teachers, schools, and school districts that maximize the outcomes of students relative to others who have the same starting point in life with regards to their genetic propensity toward completing education.…”