Rule-based and machine learning approaches for second language sentence-level readability

Abstract

We present approaches for the identification of sentences understandable by second language learners of Swedish, which can be used in automatically generated exercises based on corpora. In this work we merged methods and knowledge from machine learning-based readability research, from rule-based studies of Good Dictionary Examples and from second language learning syllabuses. The proposed selection methods have also been implemented as a module in a free web-based language learning platform. Users can use different parameters and linguistic filters to personalize their sentence search with or without a machine learning component assessing readability. The sentences selected have already found practical use as multiple-choice exercise items within the same platform. Out of a number of deep linguistic indicators explored, we found mainly lexical-morphological and semantic features informative for second language sentence-level readability. We obtained a readability classification accuracy result of 71%, which approaches the performance of other models used in similar tasks. Furthermore, during an empirical evaluation with teachers and students, about seven out of ten sentences selected were considered understandable, the rulebased approach slightly outperforming the method incorporating the machine learning model.

Vocabulary: Common or Basic?

“…Historically, work on readability is closely tied to (3), and started with finding the most frequent words (Thorndike, 1921(Thorndike, , 1931Thorndike and Lorge, 1944) with an express pedagogical purpose, both for L1 and L2 learning. While the key assumption behind this work, that learning one word is about as hard as learning another, has stood the test of time, learnability has mushroomed into a large field of research, and even a brief overview is beyond the scope of this paper-see Klare (1974) and Paasche-Orlow et al (2003) for informed but somewhat dated summaries, and for the more contemporary approach of bringing machine learning techniques to the task, see e.g., Pilán et al, 2014;Morato et al, 2021. Here we take the central idea to mean simply that effort is best spent on the words that will cover the overall distribution best, i.e., on the most common ones. Remarkably, this means that serious effort needs to be spent on function words, because these are disproportionately present in the high frequency range.…”

Manipulating the Difficulty of C-Tests

“…Another related field of research in computerassisted language learning is readability assessment and, subsequently, text simplification. There exists ample research on predicting the reading difficulty for various learner groups (Hancke et al, 2012;Collins-Thompson, 2014;Pilán et al, 2014). A specific line of research focuses on reducing the reading difficulty by text simplification (Chandrasekar et al, 1996).…”

GRAW+: A two‐view graph propagation method with word coupling for readability assessment

“…Step 1: Per-sentence reading difficulty estimation. The precise estimation of sentence-level readability is a hard problem and has recently attracted the attention of many researchers (Pilán, Volodina, & Johansson, 2014;Schumacher, Eskenazi, Frishkoff, & Collins-Thompson, 2016;Vajjala & Meurers, 2014). For efficiency, we use heuristic functions to make a rough estimation.…”