There are no wrong answers: an investigation into the assessment of candidates’ responses to essay‐based examinations

O'Donovan, Nick

doi:10.1080/03054980500222114

Cited by 8 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The likelihood of being swayed by one's own stylistic preferences in an essay-style assessment is obviously greater than when marking a structured assessment, a point that has also been noted in other types of assessment such as oral examinations [ 9 ]. The tension between rewarding sophistication and creativity in writing, whilst ensuring the test is assessing what it aims to test has been described previously in relation to the methodology of designing assessment tools [ 10 ].…”

Section: Discussionmentioning

confidence: 99%

Learning to mark: a qualitative study of the experiences and concerns of medical markers

et al. 2006

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Learning to mark: a qualitative study of the experiences and concerns of medical markers

et al. 2006

View full text Add to dashboard Cite

show abstract

“…Second, the examination paper at the centre of the study-a GCSE History paper-was chosen as an example of a fairly taxing paper from the examiner's perspective. O'Donovan (2005) notes that mark schemes in the Humanities are complex to apply as they are 'content-advisory' rather than 'content-specific'. This is in contrast to examination papers in Mathematics or the Sciences, for example, that tend to use questions that are more clearly right or wrong.…”

Section: Methodsmentioning

confidence: 98%

Online or face‐to‐face? An experimental study of examiner training

Chamberlain¹,

Taylor²

2011

Brit J Educational Tech

View full text Add to dashboard Cite

Thousands of examiners are employed to mark candidate scripts from the suite of public examinations offered to students during the compulsory and post-compulsory schooling phases in England, Northern Ireland and Wales. All examiners undergo training to ensure that they interpret correctly, and apply consistently, the mark scheme for their particular paper. Traditionally this training has been delivered face-to-face, but several benefits of an online training environment have been identified. This paper outlines the findings of an experimental study designed to quantify and compare the effects of faceto-face and online training on experienced examiners' marking reliability. A sample of 89 experienced examiners was randomly allocated to either a face-to-face or online training group. The participants marked 30 General Certificate of Secondary Education History scripts prior to receiving training and another 30 scripts following training. Two measures of reliability were calculated: marking accuracy (absolute mark differences) and consistency (rank order correlations). The findings suggest that both modes of training had comparable, statistically significant positive effects on examiners' accuracy and consistency. The evidence indicates that online training may be an effective alternative to face-to-face training for the purposes of standardising examiners' marking.

show abstract

“…A key factor is that levels-based mark schemes are often lengthy and contain a lot of information. Whilst more constrained mark schemes are associated with higher levels of reliability (Bramley, 2009;Pinot de Moira, 2013;, they do so by restricting the number of creditable responses and thus would compromise the validity of extended-response assessment (Ahmed & Pollitt, 2011;O'Donovan, 2005; Pinot de Moira, 2011a; Pinot de Moira, 2011b). An alternative is to use a holistic, rather than analytic, mark scheme.…”

Section: Mark Schemesmentioning

confidence: 99%

Investigating a new method for standardising essay marking using levels-based mark schemes

Greatorex¹,

Sutch²,

Werno³

et al. 2019

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

Standardisation is a procedure used by Awarding Organisations to maximise marking reliability, by teaching examiners to consistently judge scripts using a mark scheme. However, research shows that people are better at comparing two objects than judging each object individually. Consequently, Oxford, Cambridge and RSA (OCR, a UK awarding organisation) proposed investigating a new procedure, involving ranking essays, where essay quality is judged in comparison to other essays. This study investigated the marking reliability yielded by traditional standardisation and ranking standardisation. The study entailed a marking experiment followed by examiners completing a questionnaire. In the control condition live procedures were emulated as authentically as possible within the confines of a study. The experimental condition involved ranking the quality of essays from the best to the worst and then assigning marks. After each standardisation procedure the examiners marked 50 essays from an AS History unit. All participants experienced both procedures, and marking reliability was measured. Additionally, the participants' questionnaire responses were analysed to gain an insight into examiners' experience. It is concluded that the Ranking Procedure is unsuitable for use in public examinations in its current form. The Traditional Procedure produced statistically significantly more reliable marking, whilst the Ranking Procedure involved a complex decision-making process. However, the Ranking Procedure produced slightly more reliable marking at the extremities of the mark range, where previous research has shown that marking tends to be less reliable.

show abstract

There are no wrong answers: an investigation into the assessment of candidates’ responses to essay‐based examinations

Cited by 8 publications

References 5 publications

Learning to mark: a qualitative study of the experiences and concerns of medical markers

Learning to mark: a qualitative study of the experiences and concerns of medical markers

Online or face‐to‐face? An experimental study of examiner training

Investigating a new method for standardising essay marking using levels-based mark schemes

Contact Info

Product

Resources

About