Universal tools activation in English language proficiency assessments: A comparison of Grades 1–12 English learners with and without disabilities

Abstract

English learners (ELs) comprise approximately 10% of kindergarten to Grade 12 students in US public schools, with about 15% of ELs identified as having disabilities. English language proficiency (ELP) assessments must adhere to universal design principles and incorporate universal tools, designed to increase accessibility for all ELs, including those with disabilities. This two-phase mixed methods study examined the extent Grades 1–12 ELs with and without disabilities activated universal tools during an online ELP assessment: Color Overlay, Color Contrast, Help Tools, Line Guide, Highlighter, Magnifier, and Sticky Notes. In Phase 1, analyses were conducted on 1.25 million students’ test and telemetry data (record of keystrokes and clicks). Phase 2 involved interviewing 55 ELs after test administration. Findings show that ELs activated the Line Guide, Highlighter, and Magnifier more frequently than others. The tool activation rate was higher in listening and reading domains than in speaking and writing. A significantly higher percentage of ELs with disabilities activated the tools than ELs without disabilities, but effect sizes were small; interview findings further revealed students’ rationale for tool use. Results indicate differences in ELs’ activation of universal tools depending on their disability category and language domain, providing evidence for the usefulness of these tools.

Language assessment accommodations: Issues and challenges for the future

“…Both Kim et al (2023) and Eberharter et al (2023) highlighted the importance of a transparent and close relationship between what language users and learners experience in the language teaching/use situation, and what they encounter in the language assessment context. This is important in relation to accommodations in assessment.…”

“…It is good to see far more evidence nowadays of test providers and researchers engaging collaboratively with multiple stakeholder groups who have insights to share in this endeavour, including lived experience. The six papers in this special issue have offered an extensive list of test stakeholders who can or should be consulted, including: English language teachers (Guzman-Orth et al, 2023; Motteram et al, 2023); specialist teachers for learners with disabilities (Guzman-Orth et al, 2023); school leaders and administrators (Motteram et al); accessibility experts (Christensen et al); parents (Motteram et al); test invigilators and examiners (Guzman-Orth et al 2023; Motteram et al, 2023); and, perhaps most importantly, test takers themselves (Kim et al, 2023; Guzman-Orth et al, 2023). Guzman-Orth et al (2023), in particular, stressed the benefits of a participatory design approach that empowers collaborators and helps to co-create knowledge and understanding that are essential to successful test development.…”

“…Several papers reference the concept of universal design as the preferred theoretical foundation for language test design and development (Christensen et al, 2023; Kim et al, 2023; Guzman-Orth et al, 2023). This approach, originally derived from the field of architecture in the United States (Case, 2008), proposes a set of principles whereby assessments are intentionally and proactively designed from the earliest stage of construction to be maximally accessible to all users, regardless of any special needs they may have (cf., the planning and design of public buildings as disabled-friendly).…”