The Influence of Participants' Personality on Quantitative and Qualitative Metrics in Usability Testing

Abstract

We present the results of a usability study with 35 participants investigating the influence of personality on various metrics used in usability engineering. We conduct a task based usability test with a website integrating tasks of various difficulty and measure performance metrics like task completion rate and time on task. We also use standard questionnaire based usability metrics like the System Usability Scale. Furthermore, we gather qualitative data via open-ended questions and count the number of words as well as the mentions of positive and negative aspects. We measure personality using the well-known big five model, also often referred to as OCEAN model (openness, conscientiousness, extraversion, agreeableness, neuroticism) and three basic needs (need for influence and power, need for recognition and performance, need for security and tranquility). We analyze the relationship between personality and usability metrics via correlations and regression models. We identify multiple significant results and show that in our study the personality correlated with some of the usability metrics we inspected. Extraversion and the need for influence and power show the most and strongest correlations. Furthermore, we also show that regression models based on personality traits can explain up to 37% of the variance in usability metrics. The results have implications for the improvement of the selection process of usability test participants as well as for the interpretation of test results. We discuss these implications and give an outlook on further research in this area.ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

The relationship between perceived usability, personality traits and learning gain in an e-learning context

“…Evaluating perceived usability in relation to personality traits, research findings are quite contradictory. Schmidt et al (2019) found no significant correlations between personality traits and perceived usability score as it was reflected upon SUS. In a data set of 323 participants, Oyibo et al (2017) found that agreeableness was the strongest predictor of usability (β 5 0.16, p < 0.01), followed by conscientiousness (β 5 0.11, p 5 0.066) at 0.1 significance level; explaining together 5% of the variation of usability, Rupp et al (2018) investigated the role of individual differences regarding trust, usability and motivational impact of wearable fitness devices.…”

Using Eye Tracking to Measure Overall Usability of Online Grocery Shopping Websites

“…Since the usability assessment is needed for various tools and media, quite a lot of measures and scales have been developed for the evaluation of usability (Hartson, Andre, & Williges, 2001;Hasibuan, Santoso, Yunita, & Rahmah, 2020;Hornbaek, 2010;Madan & Dubey, 2012). Plenty of the measures proposed and used for usability evaluation are subjective self-report scales (Friedrichs, Borojeni, Heuten, Lüdtke, & Boll, 2016;Wang, Terken, & Hu, 2014); others are behavioral metrics (Arinalhaq & Widyanti, 2019;Schmidt, Wittmann, & Wolff, 2019). That said, a review of the published usability studies by Forster, Hergeth, Naujoks, and Krems (2018) shows up until 2018 preponderance of research articles merely used self-report measures for studying usability of different systems.…”