Syntactic Complexity Measures as Indices of Language Proficiency in Writing: Focus on Heritage Learners of Russian

Abstract

The concept of linguistic complexity, understood broadly as a range of basic and elaborate structures available and accessible to learners as evidenced in their production of speech and writing (Ortega, 2003), has featured prominently in second language development research since the inception of the field. The field of heritage language acquisition, however, has only recently begun to engage linguistic complexity as a comprehensive lens for studying heritage language development. The current study contributes to this fledgling area of research by investigating automatically extracted measures of syntactic complexity in the written language of heritage learners of Russian at various developmental levels. The analysis of 12 measures of syntactic complexity allows us to conclude that the majority of automatically extracted indices differentiate proficiency levels of heritage speakers in the study. The study results provide important insights into the nature of heritage language development and are readily applicable for assessment and pedagogical purposes.

Measuring Writing Development and Proficiency Gains Using Indices of Lexical and Syntactic Complexity: Evidence From Longitudinal Russian Learner Corpus Data

“…For instance, when comparing intermediate and advanced heritage learners of Russian, Kisselev et al. (2021) found that the amalgam measure of subordination (i.e., counts of all types of subordinate clauses) did not statistically differentiate these two proficiency levels in the heritage learner corpus. The authors did find that specific types of subordinate structures correlated with proficiency levels—with gerund, participle, and relative clause ratios growing with proficiency, and infinitive clauses and adverbial clause modifiers decreasing in number with proficiency.…”

“…Based on these arguments, we felt that the use of the ACTFL proficiency guidelines as a measure of proficiency was well warranted. Additionally, the ACTFL‐based proficiency assessments have been utilized in previous empirical research on L2 writing development in Russian (Henry, 1996; Kisselev & Alsufieva, 2017).…”

“…No other information on proficiency of the learners in the program is available. Two decades later, Kisselev and Alsufieva (2017) investigated writing development in L2 and heritage Russian over the course of an instructional program (3 years). All learner participants were regularly tested for proficiency according to ACTFL guidelines, and the study was able to compare the writing of intermediate and advanced writers of Russian.…”

“…Given the dearth of research on learner Russian, we selected measures that have been widely applied in research with other L2s. Next, we were constrained by the lack of computational tools for analyzing complexity in learner Russian and had to rely on tools developed for analyzing complexity in heritage Russian (Kisselev et al., 2021).…”

“…The use of CEFR levels, as well as IELTS and TOEFL scores, too have been critiqued on the grounds that they describe an overall performance, not just performance on the writing task. Nonetheless, proficiency established with the help of external, standardized tests, including the ACTFL Oral Proficiency Interview (OPI) or WPT (Kisselev & Alsufieva, 2017), currently provides the better alternative, potentially solving the problem of comparability both within the same language and cross-linguistically. Proficiency levels assigned through the standardized tests and proficiency frameworks have been demonstrated to align well.…”