Kolmogorov complexity metrics in assessing L2 proficiency: An information-theoretic approach

Abstract

Based on 774 argumentative writings produced by Chinese English as a foreign language (EFL) learners, this study examined the extent to which Kolmogorov complexity metrics can distinguish the proficiency levels of beginner, lower-intermediate, and upper-intermediate second language (L2) English learners. Kolmogorov complexity metric is a holistic information-theoretic approach, which measures three facets of linguistic complexity, i.e., overall, syntactic, and morphological complexity simultaneously. To assess its validity in distinguishing L2 proficiency, Kolmogorov complexity metric is compared with traditional syntactic and morphological complexity metrics as well as fine-grained syntactic complexity metrics. Results showed that Kolmogorov overall and syntactic complexity could significantly distinguish any adjacent pair of L2 levels, serving as the best separators explored in the present study. Neither Kolmogorov morphological complexity nor other complexity metrics at both the syntactic and morphological levels can distinguish between all pairs of adjacent levels. Results of correlation analysis showed that Kolmogorov syntactic complexity was not or weakly correlated with all the fine-grained syntactic complexity metrics, indicating that they may address distinct linguistic features and can complement each other to better predict different proficiency levels.

Utility of Kolmogorov complexity measures: Analysis of L2 groups and L1 backgrounds

“…These unique characteristics might make Kolmogorov complexity a suitable measure to examine across and within L2 speaker variation in L2 writing. Despite their strengths, information-theoretic linguistic complexity measures have rarely been utilized in the L2 writing field [ 26 ], e.g., [ 50 , 63 ].…”

“…While a sizable number of studies have explored the link between L2 proficiency and non-algorithmic syntactic complexity measures, far fewer studies have investigated this link using Kolmogorov complexity measures [ 26 , 50 ]. A pioneering study by Ehret and Szmrecsanyi [ 50 ] examined the relationship between three Kolmogorov complexity metrics and the amount of L2 instruction using the International Corpus of Learner English (ICLE).…”

Effects of speaker types and L1 backgrounds on the linguistic complexity of learners’ writing

“…Yoon's (2018) study is significant as it points out the potential applicability of morphological diversity in predicting learner proficiency. However, the morphological complexity index (MCI) adopted by Yoon (2018) was later demonstrated to be less predictable in gauging learner proficiency than the Kolmogorov morphological complexity, which exhibited a larger effect size in explaining differences across proficiency levels (Wang, Wang, & Wang, 2022).…”

“…Moreover, Wang, Wang, and Wang (2022) found that Kolmogorov overall and syntactic complexity performed best in distinguishing L2 proficiency, as compared to traditional syntactic and morphological complexity metrics as well as fine‐grained syntactic complexity metrics. This positions the Kolmogorov complexity as a robust tool, capable of predicting learner proficiency through its three layers: overall, morphological, and syntactic.…”

“…Therefore, Kolmogorov complexity is insufficient in detecting the changes of specific linguistic features compared with traditional complexity metrics. However, considering the significant effect of Kolmogorov complexity metrics in differentiating learner proficiency (Wang et al., 2022), we propose that Kolmogorov complexity metrics may complement the traditional ones to depict the variations of linguistic complexity across speaker types and L1 backgrounds. Additionally, Kolmogorov complexity's ability to evaluate overall, syntactic, and morphological complexity concurrently provides a comprehensive and multidimensional instrument for capturing the intricate nature of L2 complexity (Ehret & Szmrecsanyi, 2016).…”