A Further Report of Investigations Into the Reliability of Marking of Gce Examinations

Abstract

Summary. This paper represents an extension of a previous report (Murphy, 1978) of investigations into the reliability of marking of GCE examinations. Results are presented relating to mark‐re‐mark investigations in 20 different examination subjects carried out over a period of four years. In each case a sample of examination scripts had been re‐marked by a senior GCE examiner a short time after the normal GCE marking procedures had been completed. The results are analysed in terms of correlations between the original marks and the re‐marks. In the discussion of these results contrasts are drawn between the levels of marking reliability reported both for different examination subjects and for the different types of examination papers used within the various examinations. Finally the findings are discussed in terms of their implications for users of GCE examination grades.

Peer assessment without assessment criteria

“…Indeed, the expert inter-rater reliability was higher than might be expected for an unstructured written task assessed against criteria (e.g. Murphy 1982;Newton 1996). This finding stands in contrast to empirical results reported in the peer assessment literature, which support a correlation between the use of clearly understood criteria and good validity and reliability.…”

“…Second, expert assessments can be unreliable (Falchikov and Goldfinch 2000;Magin 2001b;Murphy 1982;Newton 1996;Topping 2003), so the assumption that they constitute an objective reference may not be warranted. To address this, our first step in this study was to measure the expert inter-rater reliability of using CJ assess advanced mathematical understanding.…”

Assessing Mathematical Problem Solving Using Comparative Judgement

“…However we did not have access to score-rescore data for the GCSE or Bowland scripts and so could not obtain inter-rater reliabilities for scoring to compare with those for CJ. For GCSE scripts we might expect scoring to achieve higher inter-rater reliabilities because of the exams being made up mostly from short, objective items (Murphy, 1982;Newton, 1996;Willmott & Nuttall, 1975).…”

A comparative judgement approach to teacher assessment

“…However, practical constraints forbade these possibilities. Reports of high-stakes examinations in science and mathematics suggest inter-rater reliabilities of up to .99 are achievable (Murphy, 1982;Newton, 1996). For lowstakes research activities involving questions designed for comparative judgement this might be lower.…”