Mitigating Satisficing in Cognitively Demanding Grid Questions: Evidence from Two Web-Based Experiments

Roßmann, Joss; Gummer, Tobias; Silber, Henning

doi:10.1093/jssam/smx020

Cited by 43 publications

(43 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the simple repetition of stimuli or of a task is also likely to lead to participants to feel repetition. These data are consistent with studies that show that grouping of multiple items per page reduce decreases in response rate, possibly by avoiding the feeling of repetition of questions, instructions and rating scales (Roßmann et al, 2018;Van Selm & Jankowski, 2006). Moreover, our data clarifies that respondents feel repetition when they find themselves doing the same task 5-6 times, or performing it for more than 6 minutes.…”

Section: Discussionsupporting

confidence: 91%

Rushing to the end: Participants’ perceptions of demotivating aspects of online surveys

Martins¹,

Lavradio²

2020

View full text Add to dashboard Cite

More and more social science studies are now acquiring data through the internet, reaching participantsonline. Some participants start out engaged and motivated to participate, but progressively slide into“rushing behaviors”. We inquired experts in survey responding about when, in online studies, theywould feel a desire for rushing (defined as speeding with no concerns about the quality of responses).This qualitative approach uncovered Repetition, Survey length and No interest in topic as the threemain features that would motivate these participants to rush in surveys. Subsequent inquiry of thesame participants indicated that repetition concerns the type of questions made (more than stimuli ortask), the execution of the same task more than 5-6 times, or for more than 6 minutes. Survey lengthconcerns a preference for shorter surveys, as well as the subjective experience in which length exceedspreviously set expectations (i.e., longer than announced), contributing to rushing by effectivelylowering the hourly pay rate as the survey increases in length. Interest in topic was reported to beconsistently low, despite not being the main reason to quit the survey. However, a change in expectedlevel of interest in the middle of the survey is reported as a factor that will promote rushing behaviors.We discuss these data as informative regarding how pre-tests of surveys can benefit from theseparticipants’ expertise.

show abstract

Section: Discussionsupporting

confidence: 91%

Rushing to the end: Participants’ perceptions of demotivating aspects of online surveys

Martins¹,

Lavradio²

2020

View full text Add to dashboard Cite

show abstract

“…Scales commonly contain either 5 or 7 columns based on the known information processing abilities of the human mind (Miller, 1994). Larger grid size, from row, columns, or a combination, may increase satisficing and cognitive load, with participants engaging in less critical processing of survey questions and their own responses (Couper, Tourangeau, Conrad, & Zhang, 2013; Roßmann, Gummer, & Silber, 2017).…”

Section: Literature Reviewmentioning

confidence: 99%

What Is the Best Size for Matrix-Style Questions in Online Surveys?

Grady

Greenspan

Liu

2018

Social Science Computer Review

View full text Add to dashboard Cite

Across two studies, we aimed to determine the row and column size in matrix-style questions that best optimizes participant experience and data quality for computer and mobile users. In Study 1 (N ¼ 2,492), respondents completed 20 questions (comprising four short scales) presented in a matrix grid (converted to item-by-item format on mobile phones). We varied the number of rows (5, 10, or 20) and columns (3, 5, or 7) of the matrix on each page. Outcomes included both data quality (straightlining, item skip rate, and internal reliability of scales) and survey experience measures (dropout rate, rating of survey experience, and completion time). Results for row size revealed dropout rate and reported survey difficulty increased as row size increased. For column size, seven columns increased the completion time of the survey, while three columns produced lower scale reliability. There was no interaction between row and column size. The best overall size tested was a 5 Â 5 matrix. In Study 2 (N ¼ 2,570), we tested whether the effects of row size replicated when using a single 20-item scale that crossed page breaks and found that participant survey ratings were still best in the five-row condition. These results suggest that having around five rows or potentially fewer per page, and around five columns for answer options, gives the optimal survey experience, with equal or better data quality, when using matrix-style questions in an online survey. These recommendations will help researchers gain the benefits of using matrices in their surveys with the least downsides of the format.

show abstract

“…However, the use of a sequential grid of institutions might also present a problem. As it has been proven by the literature on survey methodology [ 5 , 6 ], correlations (satisficing) amongst these items tend to decrease substantially when items are separated. An alternative is to present this battery of institutions in different settings.…”

Section: Survey and Experimental Design Sampling And Other Methodolomentioning

confidence: 99%

Political and affective polarisation in a democracy in crisis: The E-Dem panel survey dataset (Spain, 2018–2019)

et al. 2020

View full text Add to dashboard Cite

The E-DEM dataset provides information on the evolution of political and affective polarisation and electoral behaviour in the aftermath of the political crisis that shook the Spanish party system starting in 2014. The dataset is formed by a four-wave online panel survey of the Spanish voting age population between late October 2018 and May 2019. The four waves coincide with key moments in Spanish political life including local, regional, national, and European elections, as well as the conviction of Catalan secessionist leaders. It also covers the six-month period of the surge of Spain's new radical right party, Vox, spanning from shortly before its first major electoral success in Spain's most populous region, Andalusia, to its consolidation in the May 2019 European elections. The sample, which reflects the general population in terms of age, gender, and geographical province, consists of 1,484 panellists who completed the four waves, while the samples for individual waves are larger, ranging from 1,659 to 2,501 respondents. The data is especially useful for researchers who wish to explore dynamics of ideological and affective polarisation, factors that explain the rise of new parties, and for those investigating the evolution of political attitudes in general.

show abstract

Mitigating Satisficing in Cognitively Demanding Grid Questions: Evidence from Two Web-Based Experiments

Cited by 43 publications

References 29 publications

Rushing to the end: Participants’ perceptions of demotivating aspects of online surveys

Rushing to the end: Participants’ perceptions of demotivating aspects of online surveys

What Is the Best Size for Matrix-Style Questions in Online Surveys?

Political and affective polarisation in a democracy in crisis: The E-Dem panel survey dataset (Spain, 2018–2019)

Contact Info

Product

Resources

About