On the Limitations of Manipulation Checks: An Obstacle Toward Cumulative Science

Fayant, Marie-Pierre; Sigall, Harold; Lemonnier, Aurore; Retsin, Emilie; Alexopoulos, Théodore

doi:10.5334/irsp.102

Cited by 70 publications

(71 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In our case, due to the difficulties of creating an action efficacy manipulation check that was independent to the effect of taking online action, we argue that a manipulation check would not have added value. In making this argument we follow both Fayant, Sigall, Lemonnier, Retsin, & Alexopoulos (2017) and Sigall and Mills (1998) who argue that manipulation checks aren't necessary to establish construct validity of causes and effects. group to end violence towards migrant women'.…”

Section: Post-manipulation Measures: Participative Efficacymentioning

confidence: 99%

All click, no action? Online action, efficacy perceptions, and prior experience combine to affect future collective action

Wilkins

Livingstone

Levine

2019

Computers in Human Behavior

View full text Add to dashboard Cite

Social media is increasingly used for social protest, but does internet-enabled action lead to 'slacktivism' or promote increased activism? We show that the answer to this question depends on prior level of activism, and on beliefs about the effectiveness of individual contribution to the collective campaign. Internet-enabled action was varied quasiexperimentally, with participants (n = 143) choosing whether or not to share a campaign on social media. Participants were then informed that sharing on social media had a big (high action efficacy) or small (low action efficacy) impact on achieving the campaign's goal. Prior levels of activism were measured before the experiment, and general levels of collective action were measured one week after the experiment. Taking internet-enabled action for one campaign increased future activism for other campaignsbut only in individuals who were already active and who perceived their actions to be an effective contribution to the campaign.

show abstract

Section: Post-manipulation Measures: Participative Efficacymentioning

confidence: 99%

All click, no action? Online action, efficacy perceptions, and prior experience combine to affect future collective action

Wilkins

Livingstone

Levine

2019

Computers in Human Behavior

View full text Add to dashboard Cite

show abstract

“…Manipulation checks have been subjected to criticism for almost as long as they have been proposed (e.g., Kidd, 1976;Mills, 1969;Sigall & Mills, 1998). Indeed, some critics go so far as to argue that manipulation checks are not only unnecessary but harmful to science (Fayant et al, 2017). Many of these critiques (as we will discuss below) often depend on the manner in which a check is used and interpreted.…”

Section: Manipulation Checksmentioning

confidence: 99%

“…The measurement of manipulation checks is largely concerned with the issue of assessing construct validity -that is, the extent to which the operationalizations adequately reflect the constructs they are meant to represent (Shadish, Cook & Campbell, 2002). Some critics have expressed concerns that researchers may misunderstand exactly how manipulation checks contribute to construct validity (Fayant et al, 2017;Sigall & Mills, 1998). For instance, Sigall and Mills (1998) reasoned that measures of the construct of interest do not add any additional information on construct validity when there are no other plausible explanations for the observed effect on the dependent variable, and thus are not necessary.…”

Section: Concept and Terminologymentioning

confidence: 99%

“…Manipulation checks are a staple of modern experimental, especially social, psychology (Hauser, Ellsworth, & Gonzalez, 2018), and are considered a fundamental technique. Researchers openly assert their importance (see, e.g., Fayant, Sigall, Lemonnier, Retsin, & Alexopoulos, 2017), and they are even addressed in undergraduate-level research methods textbooks (e.g., Haslam & McGarty, 2019). Manipulation checks are purported to provide evidence of construct validity (Sigall & Mills, 1998).…”

mentioning

confidence: 99%

See 1 more Smart Citation

“Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology

Ejelöv

Luke

2020

Journal of Experimental Social Psychology

View full text Add to dashboard Cite

show abstract

“…The analysis of participants' estimates of the percentages of students against voluntary euthanasia revealed, as expected, a main effect of source status. It showed that those in the majority condition remembered that more students in the survey agreed with the message (M = 71.31) than did those in the minority condition (M = 43.14), F(1, 79) = 52.19, p < 0.001, η p 2 = 0.34 (for a discussion on the informativeness and limitations of manipulation checks, see Fayant, Sigall, Lemonnier, Retsin & Alexopoulos, 2017).…”

Section: Overviewmentioning

confidence: 99%

I Think Therefore I Am (Influenced): Perceptions of Social Influence on Self and Others

Gardikiotis

2017

rips

View full text Add to dashboard Cite

This research focuses on people's perceptions and explanations of majority and minority influence, and on the socio-cognitive processes that underlie them. Two experimental studies (Ns = 89, 139) examined the effects of source status (majority vs. minority) and message quality (strong vs. weak arguments) on perceptions of influence on self and others, explanations for this influence and message elaboration. An estimation of potential (and actual) influence on self and other recipients was examined and an expected bias in perception was evident, others were thought to be influenced more than the self (i.e. a thirdperson perception). These perceptions depended on the greater message elaboration instigated by a minority (vs. majority) source. The reasons people offer about the potential (or actual) influence were also examined -in particular, people's thoughts about the influential message and source's status (i.e. the information that they receive a message by a majority or a minority influence). Results showed that thinking as an explanation of influence is more important for a majority (vs. a minority) message and this does not depend on actual cognitive elaboration but rather on self-serving processes. Overall, the results show that thinking as an underlying process of, or as an account for, influence is differentially connected with majority and minority source. These findings contribute to our understanding of perceptions and explanations of social influence and of their underlying socio-cognitive processes.

show abstract

On the Limitations of Manipulation Checks: An Obstacle Toward Cumulative Science

Cited by 70 publications

References 25 publications

All click, no action? Online action, efficacy perceptions, and prior experience combine to affect future collective action

All click, no action? Online action, efficacy perceptions, and prior experience combine to affect future collective action

“Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology

I Think Therefore I Am (Influenced): Perceptions of Social Influence on Self and Others

Contact Info

Product

Resources

About