The Impact of Forced Answering and Reactance on Answering Behavior in Online Surveys

Abstract

Forced answering (FA) is a frequent answer format in online surveys that forces respondents to answer each question in order to proceed through the questionnaire. The underlying rationale is to decrease the amount of missing data. Despite its popularity, empirical research on the impact of FA on respondents’ answering behavior is scarce and has generated mixed findings. In fact, some quasi-experimental studies showed that FA has detrimental consequences such as increased survey dropout rates and faking behavior. Notably, a theoretical psychological process driving these effects has hitherto not been identified. Therefore, the aim of the present study was twofold: First, we sought to experimentally replicate detrimental effects of FA on online questionnaire data quality. Second, we tried to uncover an explanatory psychological mechanism. Specifically, we hypothesized that FA effects are mediated through reactance. Zero-order effects showed that FA increased state reactance and questionnaire dropout as well as reduced answer length in open-ended questions. Results of survival and mediation analyses corroborate negative FA effects on data quality and the proposed psychological process.

A generational investigation and sentiment and emotion analyses of female fashion brand users on Instagram in Sub-Saharan Africa

“…Thus, we suggest that future research employs alternatives to FA. For instance, using “soft reminders” alongside offering a “Prefer not to answer” (Sischka et al 2020 ) or “Not wish to disclose” as an additional option whilst responding to the questionnaire items could minimise missing data without elevating the risk of FA bias.…”

“…Thus, the procedure aimed for the avoidance of missing data rather than “forcing” the participants to respond. Also, our survey did not tap into personally sensitive questions (Sischka et al 2020 ). Therefore, we argue that the low dropout rate (Stieger et al 2007 ) represented by the high response rate of our survey (70%) indicates that slight bias was associated with the force answering (FA) in the current study.…”

“…Therefore, we argue that the low dropout rate (Stieger et al 2007 ) represented by the high response rate of our survey (70%) indicates that slight bias was associated with the force answering (FA) in the current study. Finally, online surveys are less susceptible to socially desirable responding (Gnambs and Kaspar 2015 ) because respondents do not have to share sensitive data in front of an interviewer (Sischka et al 2020 ).…”

Psychological Contract Violation or Basic Need Frustration? Psychological Mechanisms Behind the Effects of Workplace Bullying

“…A total of 1,609 participants (73.8%) took part and were compensated each with US $1.20. 4 No forced answering design was implemented as this has been found to be detrimental in terms of data quality (Sischka et al, 2020b ). We filtered out workers who indicated that their employment status had changed between prescreening and the actual survey (e.g., from employment to unemployment; 0.9%; n = 15).…”

Setting up Probability-Based Online Panels of Migrants with a Push-to-Web Approach: Lessons Learned from the German Emigration and Remigration Panel Study (GERPS)

“…Respondents' survey attitude and motivation to participate strongly correlate with the saliency of a survey (Groves et al 2000;Blom et al 2015;Sischka et al 2020). Survey managers are therefore interested in designing surveys that increase respondents' willingness to participate.…”