Using anchor-based methods to determine the smallest effect size of interest

Anvari, Farid; Lakens, Daniël

doi:10.1016/j.jesp.2021.104159

Cited by 115 publications

(120 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In line with the best practices for scale development to recruit at least 300 participants per sample (Boateng et al, 2018;Clark & Watson, 1995Comrey & Lee, 1992;Guadagnoli & Velicer, 1988), as well as being highly powered (power = .90, α = .05) to detect the smallest effect size of interest (r = .10, needed N = 1,046; Anvari & Lakens, 2021;Funder & Ozer, 2019;Götz, Gosling, et al, 2021), Sample 2A, 2C, and 2D exceed the size requirements. Sample 2B was highly powered (power = .90, α = .05) to detect effect sizes r of .15 (needed N = 463).…”

Section: Participantsmentioning

confidence: 82%

“…Since the MIST-20 only takes about 2 minutes to complete, we recommend researchers to use the MIST-20 whenever possible. Third, while we were sufficiently powered to detect effect sizes similar to the original evaluation of the intervention (Roozenbeek & van der Linden, 2019), we did not have sufficient power to detect smaller nuances (Anvari & Lakens, 2021;Funder & Ozer, 2019;Götz, Gosling, et al, 2021).…”

Section: Discussionmentioning

confidence: 93%

See 1 more Smart Citation

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

Maertens¹,

Fm²,

Schneider³

et al. 2021

Preprint

View full text Add to dashboard Cite

Interest in the psychology of misinformation has exploded in recent years. Despite ample research, to date there is no psychometrically validated instrument to measure people’s ability to detect misinformation. To overcome this, we introduce the Verification done framework, an overarching interpretation schema, that simultaneously considers overall veracity discernment, and the distinct, measurable abilities (real news detection, fake news detection), and biases (distrust—negative judgement bias; naïvité—positive judgement bias) that it is composed of—thus offering a holistic and nuanced assessment. We then conduct three studies with six independent samples (Ntotal = 7,291) to develop, validate, and apply the Misinformation Susceptibility Test (MIST), the first psychometrically validated measurement instrument of veracity discernment ability. In Study 1 (N = 409), we use a neural network language model to generate items for our test, and use factor analysis and item-response theory to create the MIST-20 (20 items; <2 minutes) and MIST-8 (8 items; <1 minute). In Study 2 (N = 6,461), we confirm model fit in four representative samples (US, UK), from three different sampling platforms—Respondi, CloudResearch, and Prolific. We also explore the MIST’s nomological net, which demonstrates good convergent and discriminant validity, and generate age-, region-, and country-specific norm tables. In Study 3 (N = 421), we demonstrate how the MIST can be used in practice to test the effectiveness of interventions using the Verification done framework. We provide instructions for researchers and practitioners on how to implement the MIST as a screening tool, covariate, or framework for evaluating effects.

show abstract

Section: Participantsmentioning

confidence: 82%

Section: Discussionmentioning

confidence: 93%

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

Maertens¹,

Fm²,

Schneider³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…half-width) of a 95% confidence interval for the paired difference between means (Maxwell et al, 2008). The upper 10% threshold (in units of cutaneous vascular conductance normalized to maximum vasodilatation; % CVC max ) was defined a priori as the smallest effect size of interest (SESOI; Anvari & Lakens, 2021;Gemae et al, 2021; see below for further explanation). A sample size based on a maximum precision of 5% CVC max (i.e.…”

Section: Sample Size Estimationmentioning

confidence: 99%

“…To account for possible attrition, the estimated sample size was rounded up from 14 (at 5% precision) to 16 participants. A complete description of this procedure with supporting data is provided as a supplementary file (sample_size.html; https://osf.io/kw7ag/) (Anvari & Lakens, 2021;Batterham & Atkinson, 2005;Maxwell et al, 2008;McGarr et al, 2017;Roberts et al, 2017).…”

Section: New Findingsmentioning

confidence: 99%

“…To evaluate the potential importance of between-region differences in the NO contribution to local skin heating, we identified the SESOI (i.e. the smallest between-region difference that we would not want to miss) a priori (Anvari & Lakens, 2021;Gemae et al, 2021). By examining previous research, we determined that acute ischaemic stress, ageing and chronic disease were consistently associated with alterations in the plateau response to local heating by as little as 10% CVC max (Black et al, 2008;Minson et al, 2002;Stewart et al, 2008).…”

Section: Smallest Effect Size Of Interestmentioning

confidence: 99%

See 1 more Smart Citation

Regional variation in nitric oxide‐dependent cutaneous vasodilatation during local heating in young adults

McGarr

King

Saci

et al. 2021

Experimental Physiology

View full text Add to dashboard Cite

Regional variations in cutaneous vasodilatation during local skin heating exist across the body. While nitric oxide (NO) is a well-known modulator of this response, the extent of regional differences in NO-dependent cutaneous vasodilatation during local skin heating remains uncertain. In 16 habitually active young adults (8 females; 25 ± 5 years), cutaneous vascular conductance, normalized to maximum vasodilatation (% CVC max ), was assessed at the upper chest, abdomen, dorsal forearm, thigh and lateral calf during local skin heating. Across all regions, local skin temperatures were simultaneously increased from 33 to 42 • C (1 • C per 10 s), and held until a stable heating plateau was achieved (∼40 min). Next, with local skin temperature maintained at 42 • C, 20 mM of N G -nitro-L-arginine methyl ester (L-NAME) was continuously infused at each site until a stable L-NAME plateau was achieved (∼40 min). The difference between heating and L-NAME plateaus was identified as the NO contribution for each region. There was no evidence for region-specific responses at baseline (P = 0.561), the heating plateau (P = 0.351) or L-NAME plateau (P = 0.082), but there was for the NO contribution (P = 0.048). Overall, point estimates for between-region differences in the NO contribution varied across the body from 0 to 19% CVC max . The greatest effects were observed for the abdomen, wherein the NO contribution was consistently greater than for the other regions (range: 9-19% CVC max ). The chest was consistently lower than the other regions (range: 7-19% CVC max ). The smallest effects were observed between limb regions (range: 0-2% CVC max ). These findings advance our understanding of the mechanisms influencing regional variations in the cutaneous vasodilator response to local skin heating in young adults.

show abstract

Expert opinions on the smallest effect size of interest in false memory research

Riesthuis

Mangiulli

Broers

et al. 2022

Applied Cognitive Psychology

View full text Add to dashboard Cite

In the present study, we used a new approach to establish the smallest effect size of interest (SESOI) for false memory research by asking memory researchers what they considered to be the SESOI in false memory research. They were presented with three hypothetical and three influential paper scenarios. These scenarios depicted studies examining the effects of certain manipulations (e.g., therapy) on false memory formation using well-known false memory paradigms: Deese/Roediger-McDermott, misinformation, and forced fabrication. Subsequently, they were asked for each scenario what they would consider to be the SESOI for practical and theoretical purposes and justify their decisions. We found that there was no clear consensus for the SESOI. However, memory researchers tended to accept smaller SESOIs or "any difference that leads to a p < .05," especially for theoretical ends. We argue that the lack of a general consensus is acceptable as long as proper justification is used. We discuss such rationales and provide recommendations for setting the SESOI.

show abstract

Using anchor-based methods to determine the smallest effect size of interest

Cited by 115 publications

References 65 publications

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment

Regional variation in nitric oxide‐dependent cutaneous vasodilatation during local heating in young adults

Expert opinions on the smallest effect size of interest in false memory research

Contact Info

Product

Resources

About