A Benchmark for Neural Readability Assessment of Texts in Spanish

Abstract

We release a new benchmark for Automated Readability Assessment (ARA) of texts in Spanish. We combined existing corpora with suitable texts collected from the Web, thus creating the largest available dataset for ARA of Spanish texts. All data was pre-processed and categorised to allow experimenting with ARA models that make predictions at two (simple and complex) or three (basic, intermediate, and advanced) readability levels, and at two text granularities (paragraphs and sentences). An analysis based on readability indices shows that our proposed datasets groupings are suitable for their designated readability level. We use our benchmark to train neural ARA models based on BERT in zero-shot, few-shot, and crosslingual settings. Results show that either a monolingual or multilingual pre-trained model can achieve good results when fine-tuned in language-specific data. In addition, all models decrease their performance when predicting three classes instead of two, showing opportunities for the development of better ARA models for Spanish with existing resources.

EuroCALL 2023. CALL for all Languages - Short Papers

“…Although these features are useful, they represent an impoverished view of readability. Recent years have seen a marked increase in interest in research surrounding machine-learning approaches to secondlanguage readability, with many studies focused on commonly taught languages, such as English Xu et al, 2015;Xia et al, 2016;Vajjala & Lučić, 2018), German , Russian , French (Lee & Vajjala, 2022), Italian (Azpiazu & Pera, 2019), and Spanish (Vásquez- Rodríguez et al, 2022). In addition to studying the readability of individual languages, some researchers have worked to identify language-agnostic properties that can be used for multilingual readability classification (Azpiazu & Pera, 2019).…”

A machine learning approach to Czech readability

“…Although these features are useful, they represent an impoverished view of readability. Recent years have seen a marked increase in interest in research surrounding machine-learning approaches to secondlanguage readability, with many studies focused on commonly taught languages, such as English Xu et al, 2015;Xia et al, 2016;Vajjala & Lučić, 2018), German (Hancke et al, 2012), Russian (Reynolds, 2016), French (Lee & Vajjala, 2022), Italian (Azpiazu & Pera, 2019), and Spanish (Vásquez-Rodríguez et al, 2022). In addition to studying the readability of individual languages, some researchers have worked to identify language-agnostic properties that can be used for multilingual readability classification (Azpiazu & Pera, 2019).…”