Is there a point within a self-report questionnaire where participants will start responding carelessly? If so, then after how many items do participants reach that point? And what can researchers do to encourage participants to remain careful throughout the entirety of a questionnaire? We conducted two studies (Study 1 N = 358; Study 2 N = 129) to address these questions. Our results found (a) consistent evidence that participants responded more carelessly as they progressed further into a questionnaire, (b) mixed evidence that participants who were warned that carelessness would be punished displayed smaller increases in carelessness, and (c) mixed evidence that increases in carelessness were greater within an unproctored online study (Study 1) than within a proctored laboratory study (Study 2). These findings help address when and why careless responding is likely to occur, and they suggest effective preventive strategies.
Several recent studies have examined the prevention, causes, and consequences of insufficient effort responding (IER) to surveys. Scientific progress in this area, however, rests on the availability of construct-valid IER measures. In the current paper we describe the potential merits of the page time index, which is computed by counting the number of questionnaire pages to which a participant has responded more quickly than two seconds per item (see Huang et al., 2012 ). We conducted three studies (total N = 1,056) to examine the page time index's construct validity. Across these studies, we found that page time converged highly with other IER indices, that it was sensitive to an experimental manipulation warning participants to respond carefully, and that it predicted the extent to which participants were unable to recognize item content. We also found that page time's validity was superior to that of total completion time and that the two-seconds-per-item rule yielded a construct-valid page time score for items of various word lengths. Given its apparent validity, we provide practical recommendations for the use of the page time index.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.