Developing a common scale for the assessment of writing

Hawkey, Roger; Barker, Fiona

doi:10.1016/j.asw.2004.06.001

Cited by 62 publications

(42 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…described above, it is not surprising that this analysis revealed that frequently there were no clearly defined differences between neighboring scores on the scale. Adopting a data-driven rubric development perspective (Fulcher, 2003;Hawkey & Barker, 2004), we first focused on creating a more efficient scale structure by using MFRM to determine the optimal number of steps in each scale. After deciding upon the number of steps in the rubric scale, we used intuitive processes (Fulcher, 2003;Hawkey & Barker, 2004) to adapt the scoring descriptors to reflect the levels within each rubric category.…”

Section: Rubric Analysis Revision and Re-scoring Overviewmentioning

confidence: 99%

“…Qualitative methodologies typically rely upon focus groups to provide information about the characteristic features of different levels of writing and how these should best be articulated in the rating scale. Quantitative methodologies rely upon empirical methodologies, such as Rasch measurement, to relate test taker proficiencies with rubric descriptors on an integer scale (CEF, 2001, in Hawkey & Barker, 2004.…”

Section: Rubric Developmentmentioning

confidence: 99%

See 1 more Smart Citation

Building a better rubric: Mixed methods rubric revision

2015

View full text Add to dashboard Cite

Section: Rubric Analysis Revision and Re-scoring Overviewmentioning

confidence: 99%

Section: Rubric Developmentmentioning

confidence: 99%

Building a better rubric: Mixed methods rubric revision

2015

View full text Add to dashboard Cite

“…First of all, a criteria list was written through a review of relevant literature (Raimes, 1983;Norton, 1990;Celce-Murcia, 2001;Johnson, Penny, & Gordon, 2001;Jacobs et al 1981in Weigle, 2002Weigle, 2002;Bowen and Cali, 2004;Hawkey & Barker, 2004;Darus, 2006;IELTS, 2007;Dempsey, PytlikZillig, & Bruning, 2009;Knoch, 2009). Next, 103 faculty from ELT departments from different (20) universities examined the appropriateness of the checklist considering the expressions used and the consistency between the objectives and constructs of essay writing skill and the checklist items.…”

Section: Essay Criteria Checklist (Ecc)mentioning

confidence: 99%

Measuring Essay Assessment: Intra-rater and Inter-rater Reliability

Kayapinar¹

2014

EJER

View full text Add to dashboard Cite

Problem Statement: There have been many attempts to research the effective assessment of writing ability, and many proposals for how this might be done. In this sense, rater reliability plays a crucial role for making vital decisions about testees in different turning points of both educational and professional life. Intra-rater and inter-rater reliability of essay assessments made by using different assessing tools should also be discussed with the assessment processes. Purpose of Study:The purpose of the study is to reveal possible variation or consistency in grading essay writing ability of EFL writers by the same/different raters using general impression marking (GIM), essay criteria checklist (ECC), and essay assessment scale (ESAS), and discuss rater reliability.Methods: Quantitative and qualitative data were used to present the discussion and implications for the reliability of ratings and the consistency of the measurement results. The assessing tools were applied to 44 EFL university students and 10 graders assessed the essay writing ability of the students by using GIM, ECC, and ESAS in different occasions. Findings and Results:The findings and results of the analyses indicated that using general impression marking is evidently not reliable for assessing essays. The coefficients obtained from checklist and scale assessments, considering the correlation coefficients, estimated variance components, and generalizability coefficients present valuable information, clearly show that there is always variation among the results.

show abstract

Section: Rating Scale Developmentmentioning

confidence: 99%

“…health care or the tourist industry, see Elder et al, 2013;Fulcher et al, 2011). Going beyond the research methods described above, work on L2 writing has additionally drawn on corpus linguistic techniques (Hawkey, 2001;Hawkey & Barker, 2004) to identify salient features at pre-scored writing levels so that these features could be included in the eventual writing scale.A further empirical scale development method (empirically derived, binary-choice, boundary definition scales, or EBBs) was first proposed by Turner (1995, 1999). This method requires a group of experienced teachers or raters to group performances into separate piles based on raters' perceptions of writers' relative ability, and then to identify the distinguishing features at each level.…”

mentioning

confidence: 99%

Second Language Pronunciation Assessment

Fulcher¹,

Browne²

2016

View full text Add to dashboard Cite

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms This series brings together titles dealing with a variety of aspects of language acquisition and processing in situations where a language or languages other than the native language is involved. Second language is thus interpreted in its broadest possible sense. The volumes included in the series all offer in their different ways, on the one hand, exposition and discussion of empirical findings and, on the other, some degree of theoretical reflection. In this latter connection, no particular theoretical stance is privileged in the series; nor is any relevant perspective -sociolinguistic, psycholinguistic, neurolinguistic, etc. -deemed out of place. The intended readership of the series includes final-year undergraduates working on second language acquisition projects, postgraduate students involved in second language acquisition research, and researchers, teachers and policy-makers in general whose interests include a second language acquisition component. Second Language Pronunciation Assessment SECOND LANGUAGE ACQUISITIONFull details of all the books in this series and of all our other publications can be found on http://www.multilingual-matters.com, or by writing to Multilingual Matters, St Nicholas House, 31-34 High Street, Bristol BS1 2AW, UK. SECOND LANGUAGE ACQUISITION: 107 Second Language Pronunciation AssessmentInterdisciplinary Perspectives All rights reserved. No part of this work may be reproduced in any form or by any means without permission in writing from the publisher. Edited by Talia Isaacs and Pavel Trofi movich MULTILINGUAL MATTERSThe policy of Multilingual Matters/Channel View Publications is to use papers that are natural, renewable and recyclable products, made from wood grown in sustainable forests. In the manufacturing process of our books, and to further support our policy, preference is given to printers that have FSC and PEFC Chain of Custody certification. The FSC and/or PEFC logos will appear on those books where full certification has been granted to the printer concerned.Typeset by Nova Techset Private Limited, Bengaluru and Chennai, India. Printed and bound in the UK by the CPI Books Group Ltd. Printed and bound in the US by Edwards Brothers Malloy, Inc. In Memory of Alan Davies and Danielle Guénette AcknowledgementsThis edited volume, which brings together different but complementary research perspectives to establish a common platform in which to discuss issues relevant to assessing second language (L2) pronunciation, would not have been possible without the contributions and commitment of the authors, who explore key issues through different disciplinary lenses in the chapters that make up this volume. The vision for the book arose during a cold Canadian winter at the beginning of the second decade of the 21st century, when a sense of momentum for i...

show abstract

Developing a common scale for the assessment of writing

Cited by 62 publications

References 8 publications

Building a better rubric: Mixed methods rubric revision

Building a better rubric: Mixed methods rubric revision

Measuring Essay Assessment: Intra-rater and Inter-rater Reliability

Second Language Pronunciation Assessment

Contact Info

Product

Resources

About