User Experience Evaluation Using Mouse Tracking and Artificial Intelligence

Abstract

Business platform models frequently require continuous adaptation and agility to allow new experiences to be created and delivered to customers. To understand user behavior in online systems, researchers have taken advantage of a combination of traditional and recently developed analysis techniques. Earlier studies have shown that user behavior monitoring data, as obtained by mouse tracking, can be utilized to improve user experience (UX). Many mouse-tracking solutions exist; however, the vast majority is proprietary, and open-source packages do not provide the resources and data needed to support UX research. Thus, this paper presents: 1) the development of an interaction monitoring application titled Artificial Intelligence and Mouse Tracking-based User eXperience Tool (AIMT-UXT); 2) the validation of the tool in a case study conducted on the Website of the Brazilian Federal Revenue Service (BFR); 3) the definition of a new relationship pattern of variables that determine user behavior; 4) the construction of a fuzzy inference system for measuring user performance using the defined variables and the data captured in the case study; and 5) the application of a clustering algorithm to complement the analysis. A comparison of the results of the applied quantitative methodologies indicates that the developed framework was able to infer UX scores similar to those reported by users in questionnaires.

Análise da Experiência do Usuário (UX) de Narrativa Transmídia através de Mouse-Tracking

“…Os testes de usabilidade são métodos utilizados para avaliação, centrados no usuário, e têm o objetivo de identificar como design, conteúdos e mídias podem ser melhorados durante o processo de desenvolvimento e avaliar a qualidade global da interface utilizando-se medidas de performance (PAULINO et al, 2018). Para identificar interações no site utilizamos um sistema de rastreamento de interações de usuários, a ferramenta intitulada Artificial Intelligence and Mouse Tracking-Based user Experience Evaluation Tool (AIMT-UXT), proposta por Souza et al (2019). Dados de monitoramento do comportamento do usuário obtidos pelo rastreamento do mouse podem ser utilizados para identificar problemas na navegação e interesses no conteúdo.…”

Evaluating Localized MOOCs: The Role of Culture on Interface Design and User Experience

“…According to previous studies, ASQ (The After-Scenario Questionnaire) is very popular in the taskbased UX research [41]- [43]. Meanwhile, the overall UX is usually evaluated immediately after participants complete all the interactions with the platform, and these sorts of ratings can be used to compare different designs of the platforms with similar functions [40], [44], [45]. CSUQ (The Computer System Usability Questionnaire) is proved to have the high reliability and validity for the measurement of usability in UX studies [46]- [48].…”

“…User experience is also affected by cultural differences. Souza et al [44] investigated 144 subjects on their attitude towards usability attributes of cell phones within four cultural regions. Results confirmed satisfaction and efficiency are significantly different across four cultural regions, and the outcomes are correlated with uncertainty avoidance and long/short-term orientation.…”

Metodologia Baseada em Agrupamento de Rastreamento de Mouse e Olhar para Melhoria da Experiência do Usuário

“…A ferramenta AIMT-UXT [8] foi desenvolvida para registrar as interações de usuários, a partir do uso do mouse, para posteriormente analisá-las.É composta de três módulos: Single-View, Heatmap e DataFuzzy. O software permite a coleta, organização e processamento de dados.…”

“…Este artigo apresenta um estudo de caso que comparou dados rastreados de movimentos do mouse e da atenção do olhar de um conjunto de usuários. A ferramenta AIMT-UXT [8], uma evolução do Web Tracer [9], foi aplicada em um website do governo brasileiro, durante a realização de tarefas pré-definidas, em ambiente controlado.…”