Discovering and analyzing patterns of usage to detect usability problems in web applications

Abstract

In this paper we describe two usability studies in which the interaction of users with an online application was remotely and automatically captured and analyzed. The usability studies were performed using the WebHint method for usability analysis of web applications. We evaluate two different versions of the application in order to observe the applicability of the method for successive usability analysis. The results show how the WebHint method can be used as an alternative approach to carry out successive evaluations of the usability of an application in order to analyze the evolution of different versions of its interface.

Pattern mining of user interaction logs for a post-deployment usability evaluation of a radiology PACS client

“…The log data analysis revealed four usability issues with the PACS, compared to 48 issues that were identified in the field study. Previous usability evaluations based on log data analysis also identified modest numbers of usability issues (fourteen issues with an e-mail application [14], nine with an e-learning application [13], two with the Ryanair and Journey Planner websites [16], and one with the Skyscanner and TU/e websites [16]). However, because these studies used log data analysis as the sole method of usability evaluation, they do not provide insight into the relative effectiveness of this approach.…”

“…A difference between our study and previous usability evaluations based on log data analysis is that they analyzed log data of users performing pre-defined tasks (nine tasks in [14], six in [13] and one to three in [16]), whereas we analyzed the data of users performing their daily work in an unconstrained fashion. The advantage of using pre-defined tasks is that the tasks corresponding to users' interaction patterns are known.…”

“…Several studies (e.g. [13][14][15][16][17]) have shown that usability issues could be identified based on user interaction log data, but it is unclear how effective this approach is compared to other methods of usability evaluation.…”