Usability Problems Experienced by Different Groups of Skilled Internet Users: Gender, Age, and Background

Abstract

Finding the right test persons to represent the target user group, when conducting a usability evaluation is considered essential by the HCI research community. This paper explores data from a usability evaluation with 41 participants with high IT skills, to examine if age, gender, and job function or educational background, has an impact on the amount and types of usability problems experienced by the users. All usability problems were analysed and categorised through closed coding, to group the test persons differently in relation to gender, age, and job function or educational background. The study found that the usability problems experienced across gender, age group and job function or educational background, are approximately the same. This indicates that the usual characteristics of test persons, might not be as important, and opens up for further research in regards to, if users with different skill levels, in regards to internet usage, might be more applicable.

Cultural and Linguistic Variables in Usability Testing

“…Moreover, as discussed in our population rational, Indians are used with the vehicular languages and to associate different contexts with different languages [54]. If we associate this with the findings of Billestrup et al [76] where they discuss the level of Internet expertise might be a bigger influencer, we can explain why our Indian participants had more facility with the English and overall satisfaction with the website.…”

“…The work of Billestrup et al [76] could be a better explanation as they did not find significant differences between the experience of their users across gender, age, and academic background. Instead, they explain that the results might differ more with the different level of Internet experience, which we did not evaluate in our study.…”

“…As previously discussed, we only recruited international students for our study since they already had experience using IRCC's website but were not as experienced as someone who did a permanent residency or a citizenship application process. We acknowledge having different level of expertise on the website could have influenced the results on the number of criticisms said for instance [76]. Additionally, our participants were, for the majority, enrolled in graduate programs, which indicates a higher education level than most applicants for residency in Canada [77].…”

Data-Driven Usability Test Scenario Creation

“…The purpose of performing a usability test as part of UCD is the collection of empirical data to measure usability aspects in a reliable and objective manner and to identify design problems [2,3,19]. An important element of a usability test is the test scenario: the sequence of tasks and activities executed by the test participants.…”

“…The challenge of limited test coverage is especially important in safety critical systems, where testing is done to ensure that there are no usability issues that can lead to hazardous situations [10]. Furthermore, finding the right participants to represent the target user group is essential [3], but not trivial if there exist heterogeneous subgroups. For example, test participants should be selected with different levels of training if training levels are found to strongly influence the set of product functions used by a user.…”