The BenToWeb Test Case Suites for the Web Content Accessibility Guidelines (WCAG) 2.0

Strobbe, Christophe; Koch, Johannes; Vlachogiannis, Evangelos; Ruemer, Reinhard; Velasco, Carlos A.; Engelen, Jan

doi:10.1007/978-3-540-70540-6_58

Cited by 2 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Assessment of the capability and appropriateness of accessibility evaluation tools can be carried out in at least two ways: using test suites or selecting a representative sample of websites. The former consist often of web pages that may contain accessibility issues violating specific SC (see examples of such test suites elsewhere [2,23]). Typically, these suites have several tests to assess tool behaviour with respect to a given SC: when accessibility issues have deliberately been injected tools should be able to catch these issues (producing tp); if not, that would be a fn.…”

Section: Tools Benchmarkingmentioning

confidence: 99%

Benchmarking web accessibility evaluation tools

Vigo

Brown

Conway

2013

Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility

146

View full text Add to dashboard Cite

The use of web accessibility evaluation tools is a widespread practice. Evaluation tools are heavily employed as they help in reducing the burden of identifying accessibility barriers. However, an over-reliance on automated tests often leads to setting aside further testing that entails expert evaluation and user tests. In this paper we empirically show the capabilities of current automated evaluation tools. To do so, we investigate the effectiveness of 6 state-of-the-art tools by analysing their coverage, completeness and correctness with regard to WCAG 2.0 conformance. We corroborate that relying on automated tests alone has negative effects and can have undesirable consequences. Coverage is very narrow as, at most, 50% of the success criteria are covered. Similarly, completeness ranges between 14% and 38%; however, some of the tools that exhibit higher completeness scores produce lower correctness scores (66-71%) due to the fact that catching as many violations as possible can lead to an increase in false positives. Therefore, relying on just automated tests entails that 1 of 2 success criteria will not even be analysed and among those analysed, only 4 out of 10 will be caught at the further risk of generating false positives.

show abstract

Section: Tools Benchmarkingmentioning

confidence: 99%

Benchmarking web accessibility evaluation tools

Vigo

Brown

Conway

2013

Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility

146

View full text Add to dashboard Cite

show abstract

“…In addition to these investigations, there was an initiative to create a test suite for checking new accessibility tools as they come on the market for correctness and completeness. This initiative also collected information from remote users with disabilities about the success or failure of web implementations [10]. 2 After an analysis of the functionality that was used in all of these remote data collection activities, the following sets of functional requirements were defined for a general online testing framework.…”

Section: Requirements For Managing Evaluationsmentioning

confidence: 99%

A Framework for Remote User Evaluation of Accessibility and Usability of Websites

Power

Petrie

Mitchell

2009

Universal Access in Human-Computer Interaction. Addressing Diversity

View full text Add to dashboard Cite

Abstract. The inclusion of participants that are representative of the diverse populations of users is essential for meaningful and useful evaluations of usability and accessibility on the web. This paper proposes the requirements and architecture for an automated tool suite to help manage the design and deployment of evaluations to these participants. A prototype implementation of this architecture that is being prepared is also discussed.

show abstract

The BenToWeb Test Case Suites for the Web Content Accessibility Guidelines (WCAG) 2.0

Cited by 2 publications

References 6 publications

Benchmarking web accessibility evaluation tools

Benchmarking web accessibility evaluation tools

A Framework for Remote User Evaluation of Accessibility and Usability of Websites

Contact Info

Product

Resources

About