Toward a Transparent Construct of Reading-to-Write Tasks: The Interface Between Discourse Features and Proficiency

Abstract

As a growing number of testing programs use integrated writing tasks, more validation research is needed to inform stakeholders about score use and interpretation. The current study investigates the relationship between writing proficiency and discourse features in an integrated reading-writing task. At a Middle Eastern university, 136 undergraduate students completed a reading-based writing task. The essays were holistically scored by two raters and then classified into three proficiency levels. In addition, the essays were analyzed for a number of discourse features, including fluency, lexical sophistication, syntactic complexity, grammatical accuracy, verbatim source use, and direct and indirect source use. A one-way analysis of variance was employed to look into the relationship between writing proficiency and the discourse features of interest. The results yielded significant differences across proficiency levels for a number of discourse features. Nonetheless, follow-up comparisons indicated that the differences were greater between the lowest level and the two upper levels. As for the upper levels, no statistically significant differences were found between these two levels for most of the discourse features. The implications of the study suggest that the selected discourse features play a major role at lower levels, whereas other textual features, such as cohesion, content, and organization, are more critical at higher level writing. The results also support the need in a construct of integrated writing for the inclusion of reading proficiency and knowledge about discourse synthesis.

Comparing the Lexical Features of EAP Students’ Essays by Prompt and Rating

“…Thus, the findings confirm those of previous studies, which found that prompt characteristics can affect the quality of EAP students' written texts (Cho et al, 2013;Cumming et al, 2005;Gebril & Plakans, 2013). However, there was no correlation between the essay ratings and the lexical breadth, word accessibility, or AWL usage in the students' texts, which does not support the findings of previous research (Crossley, Salsbury, et al, 2011).…”

“…However, these studies did not analyze integrated-writing tasks, but instead used independent writing tasks such as free writing and narratives. Although integrated-writing assessment studies have demonstrated that lexical features differentiate essays at rating levels (Cumming et al, 2005), other research studies have reported no difference in lexical diversity features based on essay rating (Gebril & Plakans, 2013). Furthermore, studies of prompt characteristics have shown that prompt variation in terms of the difficulty of source texts impacts the text ratings (Cho et al, 2013).…”

“…Cho, Rijmen, and Novák (2013) found that there was a relationship between ratings of integrated-writing essays from the TOEFL iBT (Internet-based Test) and characteristics of the reading and lecture prompt materials. Similarly, researchers have shown that lower proficiency L2 writers may struggle to understand the source materials provided for integrated-writing exams and may not be able to select the appropriate or relevant information to use in their essays, ultimately affecting the overall quality of their texts (Gebril & Plakans, 2013;Sawaki, Quinlan, & Lee, 2013;Wolfersberger, 2013). Whereas some integrated-writing research has shown a positive relationship between lexical sophistication (type/token ratio) and text ratings (Cumming et al, 2005), other studies have found that average word length (another measure of lexical sophistication) was not associated with essay ratings (Gebril & Plakans, 2013;Sawaki et al, 2013).…”

Second Language Pronunciation Assessment

“…More recently, with integrated tasks becoming more popular (e.g. Gebril & Plakans, 2013;Plakans, 2009), it is also important for test providers to show that different tasks in a speaking test in fact elicit different discourse types from test takers. If no differences were found, test developers would lack evidence for including extra tasks.…”

“…More recently, with integrated tasks becoming more popular (e.g. Gebril & Plakans, 2013;Plakans, 2009), it is also important for test providers to show that different…”