The effects of reading mode (digital vs printed text) on reading comprehension: A literature review of the key assessment factors

Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities

“…When analyzing responses to comprehension questions or ratings, we took into account that some participants may be more proficient than others, and some questions may be more difficult to answer (1) (3) (2) than others. To model these differences, we analyzed our data using the Rasch model, also called the one-parameter logistic model in item response theory (IRT).…”

“…Our work contributes to this line of research by studying reading speed and response time as proxies for comprehension. While there is a relatively large body of literature both in humancomputer interaction and in language assessment dealing with differences in comprehension and behavior when reading on digital devices compared to paper [1,13,29,30,57], almost no research has been conducted on how digital reading assessments need to be adapted for persons with intellectual disabilities. This is a significant research gap, given that these user groups have very different needs in terms of interface accessibility [10].…”

“…We used a subset of this parallel corpus to train a neural ATS system (fine-tuned mBART transformer model [35]) using the method described in Rios et al [48]. From the remaining documents, we selected twelve texts according to several criteria: (1) The texts should be between 100 and 600 words in length, (2) they should cover a diverse range of topics but exclude topics known to be familiar to a wide audience, and (3) the texts should not require extensive additional context for comprehension. For each of the twelve documents, we generated an automatic simplification at the A2 level using the trained model and created four multiplechoice comprehension questions.…”

“…• Responses to comprehension questions [with our assessment: correct/incorrect] • Responses to text difficulty ratings [1][2][3][4][5] • Time taken to answer each question, i.e., the total time during which the question was visible to the participant [seconds] • Reading speed when initially reading the text [words per minute, WPM] Since the ATS model sometimes does not transfer all information accurately and we designed the comprehension questions without looking at the ATS output, the correct answers in the automatic simplification could be different from the other versions. For example, the ATS model at times deleted a sentence from the original which included relevant information for answering a question, changing the correct answer for this question with respect to the automatically simplified text to "Information does not appear in the text".…”

“…Text simplification refers to the process of improving the comprehensibility of texts by reducing complexity at several linguistic levels, for instance, by using simpler vocabulary and syntactic structures, reorganizing text structure, and explaining difficult words and concepts. Primary target groups of simplified language 1 include persons with intellectual disabilities, persons with dementia, prelingually deaf persons, and non-native readers [36]. In recent years, both the demand for simplified texts and the amount of available data have been growing.…”