Thinking About Thinking Aloud: A Comparison of TwoVerbal Protocols for Usability Testing

Krahmer, Emiel; Ummelen, Nicole

doi:10.1109/tpc.2004.828205

Cited by 108 publications

(97 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most studies have found that the talk-aloud method does not alter task outcomes although it may increase the time spent on task (see Krahmer & Ummelen, 2004 for a review). We used the talk-aloud procedure to infer whether participants felt that they were becoming lost as they worked on the task.…”

Section: Talk-aloud Web Navigation Study (Ta Study)mentioning

confidence: 99%

Implicit measures of lostness and success in web navigation

Gwizdka

Spence

2007

Interacting with Computers

View full text Add to dashboard Cite

In two studies, we investigated the ability of a variety of structural and temporal measures computed from a web navigation path to predict lostness and task success. The user's task was to find requested target information on specified websites. The web navigation measures were based on counts of visits to web pages and other statistical properties of the web usage graph (such as compactness, stratum, and similarity to the optimal path). Subjective lostness was best predicted by similarity to the optimal path and time on task. The best overall predictor of success on individual tasks was similarity to the optimal path, but other predictors were sometimes superior depending on the particular web navigation task. These measures can be used to diagnose user navigational problems and to help identify problems in web site design.

show abstract

Section: Talk-aloud Web Navigation Study (Ta Study)mentioning

confidence: 99%

Implicit measures of lostness and success in web navigation

Gwizdka

Spence

2007

Interacting with Computers

View full text Add to dashboard Cite

show abstract

“…Several studies have investigated if the change in approach affects participants' performance [9,18]. In 2004, Krahmer and Ummelen [9] conducted a study with 10 participants, who performed a verbal protocol using either the original Ericson and Simon approach or the more active Boren and Ramey approach.…”

Section: Related Workmentioning

confidence: 99%

“…In 2004, Krahmer and Ummelen [9] conducted a study with 10 participants, who performed a verbal protocol using either the original Ericson and Simon approach or the more active Boren and Ramey approach. They found that the approach did not affect the number of problems detected, however there was difference in participants' performance.…”

Section: Related Workmentioning

confidence: 99%

Comparing Concurrent and Retrospective Verbal Protocols for Blind and Sighted Users

Savva

Petrie

Power

2015

Human-Computer Interaction – INTERACT 2015

View full text Add to dashboard Cite

Abstract. Verbal protocols are widely used in user studies for evaluating websites. This study investigated the effectiveness and efficiency of concurrent and retrospective verbal protocols (CVP and RVP) for both blind and sighted participants, as well as participant workload and attitudes towards these methods. Eight blind and eight sighted participants undertook both protocols in a website evaluation. RVP was more effective as measured by problems encountered for both groups, although it was no more efficient than CVP. The severity of problems identified by both protocols was equivalent. As measured on the NASA TLX, participants found RVP found more demanding than CVP. Sighted participants found rating problems during CVP more disruptive than blind participants. These results show that RVP is a more useful protocol for practitioners and researchers even though it takes more time and is more demanding for participants. It is equally applicable for both blind and sighted participants.

show abstract

“…measuring time spent on tasks) then 'think-aloud' protocol should be avoided as it may negatively impact on the performance of the users. The 'think-aloud' protocol has been generally used in achieving three goals; firstly, to find evidence for models and theories of cognitive processes; secondly, to discover and understand general patterns of behaviour in the interaction with documents or applications in order to create a scientific basis for designing; and thirdly, to test specific new documents or applications in order to troubleshoot and revise them [Krahmer & Ummelen, 2004].…”

Section: Literature Reviewmentioning

confidence: 99%

The Impact of the Combination between Task Designs and Think-Aloud Approaches on Website Evaluation

Alroobaea

Al‐Badi²,

Mayhew³

et al. 2013

JSSD

View full text Add to dashboard Cite

The growth of the Internet and related technologies has enabled the development of a new breed of dynamic website and applications that are growing rapidly in use, which have had a significant impact on many businesses. Usability testing is a method widely used for measuring the usability of a system from the perspective of real users. The main aim of this paper is to explore the efficiency of three types of usability testing method by discovering usability problems through a combination of three proposed task designs and three types of 'think-aloud' approach, and to compare the results of these methods with those of the traditional usability testing method that has been published separately. This is a missing area in usability testing. The quantitative and qualitative data are collected and analysed in order to assess the efficiency of the three combinations in terms of the usability problems encountered and their severity, and the success rate and user satisfaction of the websites being tested, through the feedback received. The research findings show clearly that the task scenarios and how they are designed and combined with different types of 'think-aloud' approaches can influence the usability testing results by discovering different types of usability problems, and can affect the results of usability measures, such as success rate, errors number and time spent.

show abstract

Thinking About Thinking Aloud: A Comparison of TwoVerbal Protocols for Usability Testing

Cited by 108 publications

References 21 publications

Implicit measures of lostness and success in web navigation

Implicit measures of lostness and success in web navigation

Comparing Concurrent and Retrospective Verbal Protocols for Blind and Sighted Users

The Impact of the Combination between Task Designs and Think-Aloud Approaches on Website Evaluation

Contact Info

Product

Resources

About