Enabling text comprehensibility assessment for people with intellectual disabilities using a mobile application

Abstract

In research on Easy Language and automatic text simplification, it is imperative to evaluate the comprehensibility of texts by presenting them to target users and assessing their level of comprehension. Target readers often include people with intellectual or other disabilities, which renders conducting experiments more challenging and time-consuming. In this paper, we introduce Okra, an openly available touchscreen-based application to facilitate the inclusion of people with disabilities in studies of text comprehensibility. It implements several tasks related to reading comprehension and cognition and its user interface is optimized toward the needs of people with intellectual disabilities (IDs). We used Okra in a study with 16 participants with IDs and tested for effects of modality, comparing reading comprehension results when texts are read on paper and on an iPad. We found no evidence of such an effect on multiple-choice comprehension questions and perceived difficulty ratings, but reading time was significantly longer on paper. We also tested the feasibility of assessing cognitive skill levels of participants in Okra, and discuss problems and possible improvements. We will continue development of the application and use it for evaluating automatic text simplification systems in the future.

Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities

“…All experiments were conducted using the Okra app ( [57]; version 0.3.1-alpha) on Apple iPads (9.7-inch). Okra is an app for conducting reading experiments on mobile touchscreen devices, and it was specifically designed for and tested with users with intellectual disabilities.…”

“…Okra is an app for conducting reading experiments on mobile touchscreen devices, and it was specifically designed for and tested with users with intellectual disabilities. For instance, it reduces the complexity of the user interface and the amount of text on screen in order to decrease the cognitive load [57].…”

“…Our work contributes to this line of research by studying reading speed and response time as proxies for comprehension. While there is a relatively large body of literature both in humancomputer interaction and in language assessment dealing with differences in comprehension and behavior when reading on digital devices compared to paper [1,13,29,30,57], almost no research has been conducted on how digital reading assessments need to be adapted for persons with intellectual disabilities. This is a significant research gap, given that these user groups have very different needs in terms of interface accessibility [10].…”