EUREKATAX: A Taxonomy for the Representation and Analysis of Qualitative Usability Test Data

Abstract

Usability tests serve as an insightful source of feedback for product teams that want to deliver user-centered solutions and enhance the User Experience (UX) of their products and services. However, in many cases, formative usability tests in particular may generate a large volume of qualitative and unstructured data that need to be analyzed for decision making and further actions. In this paper, we discuss a more formal method of analyzing empirical data, using a taxonomy, namely Engineering Usability Research Empirical Knowledge and Artifacts Taxonomy (EUREKATAX). We describe how it can provide guidance and openness for transforming fuzzy feedback statements into actionable items. The main aim of the proposed method is to facilitate a more holistic and standardized process to empirical data analysis while adapting on the solution or context. The main contributions of this work comprise the: (a) definition of the proposed taxonomy which represents an organization of information structured in a hierarchy of four main categories (discover, learn, act, and monitor), eight sub-categories, and 52 items (actions/operations with their respective properties); (b) description of a method, that is expressed through the taxonomy, and adheres to a systematic but modular approach for analyzing data collected from the usability studies for decision making and implementation; (c) formulation of the taxonomy’s theoretical framework based on meticulously selected principles like experiential learning, activity theory: learning by expanding, and metacognition, and (d) extended evaluation into two phases, with 80 UX experts and business professionals, showing on the one hand the strong reliability of the taxonomy and high perceived fit of the items in the various classifications, and on the other hand the high perceived usability, usefulness and acceptability of the taxonomy when put into practice in real-life conditions. These findings are really encouraging, in an attempt to generate comparable, generalizable and replicable results of usability tests’ qualitative data analysis, thereby improving the UX and impact of software solutions.