No abstract
This study explores whether linguistic features can predict second language writing proficiency in the Michigan English Language Assessment Battery (MELAB) writing tasks. Advanced computational tools were used to automatically assess linguistic features related to lexical sophistication, syntactic complexity, cohesion, and text structure of writing samples graded by expert raters. The findings of this study show that an analysis of linguistic features can be used to significantly predict human judgments of the essays for the MELAB writing tasks. Furthermore, the findings indicate the relative contribution of a range of linguistic features in MELAB essays to overall second language (L2) writing proficiency scores. For instance, linguistic features associated with text length and lexical sophistication were found to be more predictive of writing quality in MELAB than those associated with cohesion and syntactic complexity. This study has important implications for defining writing proficiency at different levels of achievement in L2 academic writing as well as improving the current MELAB rating scale and rater training practices. Directions for future research are also discussed.
Synthesis writing is widely taught across domains and serves as an important means of assessing writing ability, text comprehension, and content learning. Synthesis writing differs from other types of writing in terms of both cognitive and task demands because it requires writers to integrate information across source materials. However, little is known about how integration of source material may influence overall writing quality for synthesis tasks. This study examined approximately 900 source-based essays written in response to four different synthesis prompts which instructed writers to use information from the sources to illustrate and support their arguments and clearly indicate from which sources they were drawing (i.e., citation use). The essays were then scored by expert raters for holistic quality, argumentation, and source use/inferencing. Hand-crafted natural language processing (NLP) features and pre-existing NLP tools were used to examine semantic and keyword overlap between the essays and the source texts, plagiarism from the source texts, and instances of source citation and quoting. These variables along with text length and prompt were then used to predict essays scores. Results reported strong models for predicting human ratings that explained between 47 and 52% of the variance in scores. The results indicate that text length was the strongest predictor of score but also that more successful writers include stronger, semantically-related information from the source, provide more citations and do so later in the text, and copy less from the text. This work introduces the use of NLP techniques to assess source integration, provides details on the types of source integration used by writers, and highlights the effects of source integration on writing quality.
Reading comprehension relies on a variety of complex skills that are not effectively assessed by existing Russian language tests. At the same time, Russian textbooks are criticized both for their low text quality and high text complexity. This study addresses issues of Russian language proficiency and comprehension assessment with the development of the Russian Language Test (RLT). The RLT was constructed to measure proficiency relevant to textbook comprehension, such as grammar and vocabulary knowledge, establishing propositional meaning and inferencing. Results from this initial study including 81 fifth-grade and 94 ninth-grade students confirm that students struggle with grammatical inferences and identifying the main idea in a text. Additionally, three standardized Russian exams, VPR, OGE, EGE are analyzed, affording an overview of the testing system for the Russian language from the elementary through high school education levels. This study demonstrates promise for the use of the RLT as a language proficiency assessment and provides a broad context for understanding the current state of Russian language tests for native speakers.
The comparative and combined contributions of N-grams, Coh-Metrix indices, and error types in the L1 classification of learner texts. In S. Jarvis and S. A. Crossley (Eds.), Approaching language transfer through text classification: Approaching Language Transfer through Text Classification: Explorations in the Detection-Based Approach.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.