Developing a scoring rubric for L2 summary writing: a hybrid approach combining analytic and holistic assessment

Yamanishi, Hiroyuki; Ono, Masumi; Hijikata, Yuko

doi:10.1186/s40468-019-0087-6

Cited by 21 publications

(31 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are encouraged to prepare several sample responses beforehand and share rubrics with students, so that the challenges the raters in this study faced can be mitigated and a clearer connection between teaching and assessment. Second, teachers need to pay particular attention to verbatim source use in terms of quality and quantity of paraphrasing (see Yamanishi, Ono, & Hijikata, 2019) when scoring integrated compositions. To begin with, teachers and raters should have a clear idea about variation of appropriate source use strategies across integrated writing tasks before implementing a task.…”

Section: Discussionmentioning

confidence: 99%

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

2019

View full text Add to dashboard Cite

Integrated writing tasks are becoming popular in the field of language testing, but it remains unclear how teachers assess integrated writing tasks holistically and/or analytically and which is more effective. This exploratory study aims to investigate teacher-raters' holistic and analytic ratings for reliability and validity and to reveal their perceptions of grading the integrated writing task on the Test of English as a Foreign Language Internet-based Test (TOEFL iBT). Thirty-six university students completed a reading-listening-writing task. Seven raters scored the 36 compositions using both a holistic and an analytic scale, and completed a questionnaire about their perceptions of the scales. Results indicated that the holistic and analytic scales exhibited high inter-rater reliability and there were high correlations between the two rating methods. In analytic scoring, which contained four dimensions, namely, content, organization, language use, and verbatim source use, the dimensions of content and organization were highly correlated to the overall analytic score (i.e., the mean score of the four dimensions). However, the dimension of verbatim source use was found to be peculiar in terms of construct validity for the analytic scale. The analyses also indicated various challenges the raters faced while scoring. Their perceptions varied particularly regarding verbatim source use: Some raters tended to emphasize the intricate process of textual borrowing while others stressed the difficulty in judging multiple types and degrees of textual borrowing. Pedagogical implications for the selection and use of rubrics as well as the teaching and assessment of source text use are suggested.

show abstract

Section: Discussionmentioning

confidence: 99%

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

2019

View full text Add to dashboard Cite

show abstract

“…The second one tested or writing performance test used to find out the students' performance in writing and the rubric used is an analytic score based on content, organization, grammar, vocabulary, and mechanics. From score per component, it is calculated by adding four component scores into overall (Yamanishi, Ono, & Hijikata, 2019;Beyrelï & Ari, 2009).…”

Section: Methodsmentioning

confidence: 99%

EFL Students' Perspectives on the Use of Blog towards Their Writing Performance

2020

View full text Add to dashboard Cite

Obtaining English Foreign Language (EFL) students’ views dealing with their preferences of blog to assist their writing skills in practice is needed in order to make them more helpful in writing practice. This research aims at finding out significant correlation between students’ perspective on the use blog in writing activity and their writing performance. It is correlational study that correlates two variables: blog use and writing performance. The result of this research shows that there is no significant correlation between the students’ perception on blog use in writing activities with their writing performances. Although the students have high and positive perception toward blog use in writing actives done by the teacher in the class, it does not give high response to the students’ writing performance. It happens because some extraneous and possible factors may influence the result of this research. Finally, this research is expected to contribute towards enriching teaching approach knowledge in writing.

show abstract

“…Summarization is considered one of the most important academic skills in tertiary education (Hirvela & Du, 2013;Yamanishi et al, 2019). University students are often asked to summarize articles or source materials they read in their English courses (Keck, 2006;2014;Kirkland & Saunders, 1991;Norris, 2007), whereas English teachers use summarization to evaluate their students' comprehension and writing skills (Chiu et al, 2013;Westby et al, 2010).…”

Section: Review Of Literaturesmentioning

confidence: 99%

“…However, university students do not receive adequate instruction on summary writing in elementary and secondary education in Japan. In recent years, the Ministry of Education, Culture, Sports, Science, and Technology (MEXT) in Japan compelled English teachers in secondary schools to teach integrated language tasks such as English summary writing (MEXT, 2018;Yamanishi et al, 2019). However, it was still the teachers' prerogative to give frequent assignments or decipher guidelines for summarization as a learning strategy based on their students' academic ability.…”

Section: Review Of Literaturesmentioning

confidence: 99%

Summarization in English as a Foreign Language: A Study Comparing Summary Performances to Summarizers’ Vocabulary Size

Kato¹

2021

ELT

View full text Add to dashboard Cite

The study examined the impact of a first language’s summarizing skill and second language vocabulary size on summary performances in a second language. A total of 40 English as a Foreign Language (EFL) learners from a Japanese university with a mixed level of English language proficiency were asked to write a summary in English (i.e., their non-native language, L2) and in Japanese (their native language, L1) from a text written English and Japanese respectively. The effect of L1 summarizing skill on L2 summary performances was examined using multiple regression analysis. L1 summary performances (i.e., summarizing skill) slightly influenced English summary performances for summary writers with lower-level English language proficiency but not L2 summary performances for those with higher-level English language proficiency. The participants’ vocabulary size measured by Nation’s (2007) test was positively correlated with their English summary performances. Moreover, the results showed that the vocabulary size in the highest and smallest-vocabulary size groups was correlated with scores on two rating scales (i.e., Language use and Source use) in their English summary. In contrast, the vocabulary size in the middle-level vocabulary size groups was correlated with their scores on two different rating scales (i.e., Main idea coverage and Integration) in their English summary. This study concluded that L1 summary performance had not impact on L2 summary performances because several characteristics influence of summary writers’ English vocabulary size. The study made several recommendations to EFL teachers who teach summary writing and for further study.

show abstract

Developing a scoring rubric for L2 summary writing: a hybrid approach combining analytic and holistic assessment

Cited by 21 publications

References 36 publications

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

Holistic and Analytic Assessments of the TOEFL iBT® Integrated Writing Task

EFL Students' Perspectives on the Use of Blog towards Their Writing Performance

Summarization in English as a Foreign Language: A Study Comparing Summary Performances to Summarizers’ Vocabulary Size

Contact Info

Product

Resources

About