What happened to remote usability testing?

Andreasen, Morten Sieker; Nielsen, Henrik Villemann; Schrøder, Simon Ormholt; Stage, Jan

doi:10.1145/1240624.1240838

Cited by 128 publications

(89 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These results are consistent with those of Tullis et al [39]. In a recent study [41] three methods for remote usability testing and a traditional laboratory based think-aloud method were compared. The three remote methods were a remote synchronous condition, where testing was conducted in real time but the usability evaluator was separated spatially from the test participants, and two remote asynchronous conditions, where the usability evaluator and the test subjects were separated both spatially and temporally.…”

Section: Remote Usability Evaluationsupporting

confidence: 89%

Usability testing: a review of some methodological and technical aspects of the method

Bastien¹

2010

International Journal of Medical Informatics

290

194

View full text Add to dashboard Cite

The aim of this paper is to review some work conducted in the field of user testing that aims at specifying or clarifying the test procedures and at defining and developing tools to help conduct user tests. The topics that have been selected were considered relevant for evaluating applications in the field of medical and health care informatics. These topics are: the number of participants that should take part in a user test, the test procedure, remote usability evaluation, usability testing tools, and evaluating mobile applications.

show abstract

Section: Remote Usability Evaluationsupporting

confidence: 89%

Usability testing: a review of some methodological and technical aspects of the method

Bastien¹

2010

International Journal of Medical Informatics

290

194

View full text Add to dashboard Cite

show abstract

“…In the cooperative usability testing [26] the user is invited to review the task solving process upon its completion and to reflect on incidents and potential usability problems. In asynchronous remote usability testing the user may be required to self-report incidents or problems, as a substitute of having these identified on the basis of interaction data [27].…”

Section: Uems For Users' Design Feedbackmentioning

confidence: 99%

“…This is, in particular, seen in the five studies of usability testing with self-reports where the users' design feedback consisted mainly of reports of problems or incidents [27,44,[46][47][48]. Here, the users were to run the usability test and report on the usability problems independently of the test administrator, potentially saving evaluation costs.…”

Section: The Budget Approachmentioning

confidence: 99%

“…For example, in their study of usability testing with disabled users, Petrie et al [48] compared the selfreported usability problems from users that self-administer the usability test at home to those that participate in a similar usability test in the usability laboratory. Likewise, Andreasen et al [27], Bruun et al [44] compared different approaches to remote asynchronous usability testing. In these studies of self-reported usability problems, users' design feedback hardly generated findings that complemented other data sources.…”

Section: The Budget Approachmentioning

confidence: 99%

See 1 more Smart Citation

Users’ design feedback in usability evaluation: a literature review

Følstad

2017

Hum. Cent. Comput. Inf. Sci.

View full text Add to dashboard Cite

IntroductionInvolving users in usability evaluation is valuable when designing information and communication technology (ICT), and a range of usability evaluation methods (UEM) support user involvement. Relevant methods include adaptations of usability testing [1], usability inspection methods such as pluralistic walkthrough [2], and inquiry methods such as interviews [3], and focus groups [4].Users involved in usability evaluation may generate two types of data. We term these interaction data and design feedback. Interaction data are recordings of the actual use of an interactive system, such as observational data, system logs, and data from think-aloud protocols. Design feedback are data on users' reflections concerning an interactive system, such as comments on experiential issues, considerations of the system's suitability for its context of use, usability problem predictions, and design suggestions. AbstractAs part of usability evaluation, users may be invited to offer their reflections on the system being evaluated. Such reflections may concern the system's suitability for its context of use, usability problem predictions, and design suggestions. We term the data resulting from such reflections users' design feedback. Gathering users' design feedback as part of usability evaluation may be seen as controversial, and the current knowledge on users' design feedback is fragmented. To mitigate this, we have conducted a literature review. The review provides an overview of the benefits and limitations of users' design feedback in usability evaluations. Following an extensive search process, 31 research papers were identified as relevant and analysed. Users' design feedback is gathered for a number of distinct purposes: to support budget approaches to usability testing, to expand on interaction data from usability testing, to provide insight into usability problems in users' everyday context, and to benefit from users' knowledge and creativity. Evaluation findings based on users' design feedback can be qualitatively different from, and hence complement, findings based on other types of evaluation data. Furthermore, findings based on users' design feedback can hold acceptable validity, though the thoroughness of such findings may be questioned. Finally, findings from users' design feedback may have substantial impact in the downstream development process. Four practical implications are highlighted, and three directions for future research are suggested. RESEARCHFølstad Hum. Cent. Comput. Inf. Sci. (2017) Cent. Comput. Inf. Sci. (2017) 7:19 The value of interaction data in evaluation is unchallenged. Interaction data is held to be a key source of insight in the usability of interactive systems and has been the object of thorough scientific research. Numerous empirical studies concern the identification of usability problems on the basis of observable user behaviour [5]. Indeed, empirical UEM assessments are typically done by comparing the set of usability problems identified through the assessed UEM with a ...

show abstract

“…This has driven businesses and academics to consider remote usability evaluation as an "increasingly important alternative to conventional usability evaluation" [3].…”

Section: Introductionmentioning

confidence: 99%

Application of CAUX in Remote Evaluation of Websites

Wang¹,

Liu

2018

Proceedings of Information Science and Cloud Computing — PoS(ISCC 2017)

View full text Add to dashboard Cite

In order to address problems of recording data in a complex way and utilizing unreal scene in traditional methods, the author proposes a method about remote evaluation of websites by using the Context-Awareness User Experience tool. This method helps researchers get the user data from real environment. The CAUX tool can collect user data automatically without disturbance, and it can also make semi-automated analysis of the data to reduce the burden on UX researchers. The CAUX tool needs to be installed in the user's mobile phone in advance. It can collect data of the situation about users according to the collection rules set by the UX researcher. The user data, which include objective data, URL sequences, screenshot etc, can be uploaded to the server database automatically for storage. These data can also be presented in a visual interface. Then UX researchers analyse them to figure out usability problems. In this paper, the author applies the CAUX tool to a case to study the feasibility of the remote user experience evaluation method. The case is selected as the website of Dalian maritime university. The results indicate that the application of CAUX tool in analysing user experience about remote mobile site can be more realistic and objective. Also, the data are shown in a form of visualization so as to improve UX researchers data analysis efficiency and help UX researchers conduct user experience research more effectively. ISCC2017

show abstract

What happened to remote usability testing?

Cited by 128 publications

References 24 publications

Usability testing: a review of some methodological and technical aspects of the method

Usability testing: a review of some methodological and technical aspects of the method

Users’ design feedback in usability evaluation: a literature review

Application of CAUX in Remote Evaluation of Websites

Contact Info

Product

Resources

About