Is cross‐lingual readability assessment possible?

Abstract

Most research efforts related to automatic readability assessment focus on the design of strategies that apply to a specific language. These state‐of‐the‐art strategies are highly dependent on linguistic features that best suit the language for which they were intended, constraining their adaptability and making it difficult to determine whether they would remain effective if they were applied to estimate the level of difficulty of texts in other languages. In this article, we present the results of a study designed to determine the feasibility of a cross‐lingual readability assessment strategy. For doing so, we first analyzed the most common features used for readability assessment and determined their influence on the readability prediction process of 6 different languages: English, Spanish, Basque, Italian, French, and Catalan. In addition, we developed a cross‐lingual readability assessment strategy that serves as a means to empirically explore the potential advantages of employing a single strategy (and set of features) for readability assessment in different languages, including interlanguage prediction agreement and prediction accuracy improvement for low‐resource languages.

Supervised and Unsupervised Neural Approaches to Text Readability

“…We are aware of one study that explored the transferability of these formulas across genres (Sheehan, Flor, and Napolitano 2013), and one study that explored transferability across languages (Madrazo Azpiazu and Pera 2020). The study by Sheehan, Flor, and Napolitano (2013) concludes that, mostly due to vocabulary specifics of different genres, traditional readability measures are not appropriate for cross-genre prediction, because they underestimate the complexity levels of literary texts and overestimate that of educational texts.…”

“…The study by Sheehan, Flor, and Napolitano (2013) concludes that, mostly due to vocabulary specifics of different genres, traditional readability measures are not appropriate for cross-genre prediction, because they underestimate the complexity levels of literary texts and overestimate that of educational texts. The study by Madrazo Azpiazu and Pera (2020), on the other hand, concludes that the readability level predictions for translations of the same text are rarely consistent when using these formulas.…”

Textual Analysis in Accounting: What's Next?*

“…These studies use readability scores assigned by human annotators to train models to predict the readability of unseen text. For example, Petersen and Ostendorf (2009), Vajjala and Meurers (2012), and Madrazo Azpiazu and Pera (2020) use traditional machine learning models to characterize text as a set of readability features that include (i) conventional features, such as FOG; (ii) discourse features that measure text coherence and cohesion; and (iii) features such as how commonly used a word is. Filighera et al (2019) demonstrate high performance of ANN‐based word embeddings to measure readability.…”

MultiAzterTest: a Multilingual Analyzer on Multiple Levels of Language for Readability Assessment

“…In multilingual approaches that also analyze Basque, English and Spanish, shallow, morphological, syntactic and semantic features have in taken into account (Madrazo Azpiazu and Pera, 2020). In this case, different parsers have been used: Freeling for English, Spanish, French, Catalan, and Italian (Padró et al, 2010) and Katea for Basque (Bengoetxea and Gojenola, 2010).…”