Measuring web usability using item response theory: Principles, features and opportunities

Abstract

a b s t r a c tUsability is considered a critical issue on the web that determines either the success or the failure of a company. Thus, the evaluation of usability has gained substantial attention. However, most current tools for usability evaluation have some limitations, such as excessive generality and a lack of reliability and validity. The present work proposes the construction of a tool to measure usability in e-commerce websites using item response theory (IRT). While usability issues have only been considered in theoretical or empirical contexts, in this study, we discuss them from a mathematical point of view using IRT. In particular, we develop a standardised scale to measure usability in e-commerce websites. This study opens a new field of research in the ergonomics of interfaces with respect to the development of scales using IRT.

A Comparison of Usability Testing Methods for an E-Commerce Website: A Case Study on a Malaysia Online Gift Shop

“…Our findings however shed some light over Elling et al's [21] expectation of RTE whereby it improves the interaction between subjects and moderator in terms of difficulties. Table 2 Defect count for each method FCAT RTA RTE 6 10 18…”

“…Usability of a website is critical in determining the success or failure of a company [6]. However, many ecommerce applications still do not meet customers' usability requirements [2,7].…”

Psychometric properties of the complete version of the World Health Organization Quality of Life Assessment (WHOQOL-OLD): reduced response scale

“…However, according to the author (Baker & Kim, 2017), these indicators were proposed for dichotomous items, and cutoff points considered for these parameters need to increase because of the number of options on the studied scale. There is a need to consider the tendency of increased discrimination of items seen with a higher number of possible answers because, naturally, these respondents would have more chances to be divided according to answer options (Cavalcanti, Melo, Medeiros, Oliveira, & Gouveia, 2016;Tezza, Bornia, & Andrade, 2011). For this reason, comparison of discrimination of scales is sensible because they contain items with a number of different answer options.…”

Benchmarking by Item Response Theory (BIRTH)

“…No Brasil, as avaliações do ensino básico, por intermédio do Sistema Nacional de Avaliação da Educação Básica (SAEB), começaram em 1995 a utilizar a TRI para acompanhar a evolução das 4 as e 8 as séries (atualmente 5º e 9º anos) do ensino fundamental e a 3ª série do ensino médio (SAEB, 2001, KLEIN, 2003. Outras áreas de conhecimento já aplicam a TRI, cita-se a nutrição (WILDE; GERALD; FRIEDMAN, 2004), como, por exemplo, na forma de alimentação, qualidade de vida, na medicina (YESAVAGE; BRINK; ROSE., 1983, WARE;BJORNER;KOSINSKI, 2000, MESBAH;COLE;LEE, 2002;DAS;HAMMER, 2005), bem como nas avaliações do nível de stress, na administração (COSTA, 2001, BAYLEY, 2001, BORTOLOTTI, 2003, BORTOLOTTI, 2010, para a medição da satisfação dos clientes e na própria engenharia de produção (ALEXANDRE et al, 2002a, ALEXANDRE et al, 2002bBORNIA;ANDRADE, 2011), no tocante à adoção de práticas de qualidade.…”