The concurrent think-aloud method is a highly valued technique that is used to understand peoples' taskbased cognitive processes, within both usability testing and the broader study of human-computer interaction. We investigated whether an explicit explanation-based think-aloud instruction leads to differences in navigation performance over the classic think-aloud method. The results showed an interaction between think-aloud instructions and task difficulty. For the low difficulty tasks, there was no difference in task success or in the number of link traversals between the two think-aloud conditions. However, for the more difficult tasks there were differences: participants in the classic condition completed fewer tasks successfully and engaged in more link traversals than participants in the explicit think-aloud condition. A qualitative analysis of participants' verbal data showed that both think-aloud conditions were dominated by procedural descriptions. The explicit think-aloud did yield proportionally more explanatory utterances, however the performance differences in respect of task success and link traversal suggest that an explicit think-aloud may threaten test validity and therefore evaluators must exercise caution in using this technique.
We present the results of a study that compared two placements of the Retrospective Think-aloud (RTA): A Post-session RTA where the think-aloud occurs after all tasks are complete, and a Post-task RTA where the think-aloud is elicited after each task. Data from task performance and verbal measures were collected from 24 participants. The results suggest that in terms of task performance, participants in the Post-session RTA condition performed tasks faster, with fewer errors and fewer clicks than in the Post-task RTA condition. In terms of utterances, participants in the Post-task RTA condition produced significantly more utterances that explained actions, expectations and procedural descriptions than in the Post-session RTA condition.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.