Techniques for observing selection and reading behavior in professional documents, such as the thinking-aloud and the click-and-read methods, may affect the reading process to be observed. Such so-called reactivity problems complicate the use of these instruments in experimental research and usability testing. If their influence is unknown, any experimental results obtained with these instruments may be caused by the testing method. One way to detect reactivity effects is to compare different instruments in a series of experimental studies. In this initial study, we compared the thinking-aloud method, the click-and-read method, a combination of these two methods, and a silent reading condition. Subjects read and judged a 53-page policy document in one of these conditions. We investigated whether or not different observation instruments caused specific differences in information selection, judgment, and knowledge. Thinking aloud did not cause any differences in the selection of information. Both the thinking-aloud and the click-and-read methods affected the judgment task outcome. Thinking aloud led to many positive and few negative judgments, whereas silent reading led to many negative and few positive judgments. The results for the click-and-read method showed a tendency toward the same effect. Neither method affected the knowledge test results.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.