A case-study of the predictive validity and reliability of Key Stage 2 test results, and teacher assessments, as baseline data for target-setting and value-added at Key Stage 3

Abstract

This case-study investigates the predictive validity and reliability of Key Stage 2 test results, and teacher assessments, for target-setting and value-added assumptions at Key Stage 3. (In England Key Stage 2 tests are taken in the core subjects of English, Mathematics and Science at the age of 11. Key Stage 3 tests are taken in the same subjects at the age of 14. Teacher assessments are also completed for these subjects at both key stages.) The study employed the type of linear regression analysis recommended in several government reports, to correlate Key Stage 2 test results, and teacher assessments, in core subjects, with Key Stage 3 test results, and teacher assessments, in both core and non-core subjects. Following government recommendations that the use of any other form of testing -such as the National Foundation for Educational Research (NFER) Cognitive Abilities Test (CAT) -was now no longer necessary to provide baseline data for value-added calculations, or to set targets, correlations were also investigated between results on the CAT, and test results and teacher assessments at Key Stage 3, for both core and non-core subjects, to see whether this recommendation was well founded. The results of the case-study suggest that Key Stage 2 data, both in the form of test results and teacher assessments, have little or no predictive validity, or reliability, for test results or teacher assessments at Key Stage 3. the predictive validity for non-core subjects at Key Stage 3 was so low as to be negligible. However, the CAT average score correlated more highly with both teacher assessments and test results at Key Stage 3 in core subjects, although this relationship was not re ected in non-core subjects. These ndings suggest that the predictive validity and reliability of Key Stage 2 data is seriously open to question as baseline data for either value-added, or target-setting procedures, at Key Stage 3. It should be pointed out, however, that these ndings are provisional, since they are based on data from two intake years, but preliminary analysis of data from a further three intake years appears to indicate that the concerns identi ed are well founded. KEY WORDS predictive validity; predictive reliability; baseline data; target-setting; value-added; prior attainment.

SATs, sets and allegations of bias: The allocation of 11‐year‐old students to mathematics sets in some English schools in 2015. A response to Connolly et al., 2019

“…The body responsible for National Curriculum tests advises against over‐confidence in SAT scores as a basis for accurate predictions of individual future performance (OfQual, 2013c). Research confirms this (Benton & Sutch, 2014; Doyle & Ray, 2005; Moody, 2010; Newton, 2009; Smith, 2013; Thomson, 2019; Treadaway, 2019). None of this is surprising given that national assessments had been primarily designed to assess schools, not students (Bew, 2011).…”

Investigating the reliability of the Key Stage 2 test results for assessing individual pupil achievement and progress in England

“…They have been shown to be more reliable than Key Stage test data for both predictive purposes at the end of Key Stage 3 and for value added purposes (Moody, 2001), yet in 1997, the Schools Curriculum and Assessment Authority (SCAA), the forerunner of the QCA, had told schools that the CATs were unnecessary because the Key Stage 2 data provided reliable predictors for end of Key Stage 3 achievement. Even in 2002 (Ofsted, 2002) the use of CATs was still considered unnecessary because 'it represents considerable duplication of effort when Year 6 pupils have already been assessed in most aspects of the core subjects at the end of Key Stage 2' (p. 2).…”