Development and trial of an educational tool to support the accessibility evaluation process

Abstract

This paper describes the design and development of a web accessibility knowledge management tool, known as the Accessibility Evaluation Assistant (AEA) designed to assist novice auditors in the process of an accessibility evaluation. The software incorporates a structured walkthrough approach to guide the auditor through a series of checks for established accessibility principles with the goal of conducting a comprehensive accessibility evaluation. The tool also offers the functionality to tailor the evaluation and prioritise checks based on the needs of two different user groups, or the specific content features of the website. The tool has recently been trialled with a group of 38 undergraduate computing students studying an Accessibility and Adaptive Technologies module with the aim of assessing its reliability and accuracy to validate the method. Some initial conclusions about the reliability and validity of the method and the pedagogical implications of the tool are presented. The results will help highlight the checks that can easily be verified by novices, and those that require a more detailed understanding of accessibility; require informed judgement; or are open to individual interpretation.

An approach to improve the accessibility and usability of existing web system

“…Despite the limitations of the Web Content Accessibility Guidelines (WCAG), such as the difficulty to understand the language that describes the guidelines [3], they are the dominant approach to support developers building accessible systems [11].…”

Evaluation of the effectiveness of a tool to support novice auditors

“…Unlike WCAG 1.0 and 2.0 this priority level is not fixed but changes depending on the relative potential impact it could have on that user group. For a full list of User Groups in the AEA and a description of the priority levels see [8].…”

“…Expertise in accessibility is required to fully appreciate contextual factors. An explanation of how this work supports novices in appreciating contextual factors is provided in [7], [8].…”

“…A previous trial examined the pedagogical potential of the AEA when incorporated into the undergraduate computing curriculum [20] and a comparison of the results of novice and expert evaluations yielded promising levels of validity and reliability [8].…”

Measuring and comparing the reliability of the structured walkthrough evaluation method with novices and experts

“…This helps to encapsulate the expertise and introduce the novice to the fundamental principles of accessible design. The AEA has previously been trialled with computing students and those experiments found that evaluations produced by novices using the Structured Walkthrough method have consistent levels of reliability and promising levels of validity when compared to an evaluation produced by an expert [4]. The evaluations produced by novices using the Structured Walkthrough method when compared to a WCAG 2.0 Conformance Review was found to be more effective, with higher levels of reliability and validity.…”