Further Evidence for the Cliff Effect in the Interpretation of Levels of Significance

Rosenthal, Robert; Gaito, John

doi:10.2466/pr0.1964.15.2.570

Cited by 57 publications

(52 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As in Rosenthal & Gaito's (1963) study, the degree Of belief was dways greater for n = 100 than for n = 10. As in Rosenthal & Gaito's (1963) study, the degree Of belief was dways greater for n = 100 than for n = 10.…”

Section: Resultssupporting

confidence: 61%

“…In one of the first experiments on the use of significance tests (Rosenthal & Gaito, 1963Beauchamp & May, 1964;Nelson, Rosenthal & Rosnow, 1986), researchers in psychology were asked to state their degree of belief in the hypothesis of an effect as a function of the associated p-values and sample sizes. In one of the first experiments on the use of significance tests (Rosenthal & Gaito, 1963Beauchamp & May, 1964;Nelson, Rosenthal & Rosnow, 1986), researchers in psychology were asked to state their degree of belief in the hypothesis of an effect as a function of the associated p-values and sample sizes.…”

Section: The Uses Of Null Hypothesis Significance Testing Revisited Fmentioning

confidence: 99%

“…Experiment 1 consisted in a repiication of the aforementioned experiment by Rosenthal & Gaito (1963) on the interpretation of p-values. They investigate the role of the various "ingredients" that are commonIy available in publications for interpreting statistical results.…”

Section: The Uses Of Null Hypothesis Significance Testing Revisited Fmentioning

confidence: 99%

See 2 more Smart Citations

Uses, Abuses and Misuses of Significance Tests in the Scientific Community: Won't the Bayesian Choice Be Unavoidable?

Lecoutre¹,

Lecoutre²,

Poitevineau³

2001

International Statistical Review / Revue Internationale de Stat

View full text Add to dashboard Cite

Summary The current context of the “significance test controversy” is first briefly discussed. Then experimental studies about the use of null hypothesis significance tests by scientific researchers and applied statisticians are presented. The misuses of these tests are reconsidered as judgmental adjustments revealing researchers’requirements towards statistical inference. Lastly alternative methods are considered. Consequently we automatically ask ourselves “won't the Bayesian choice be unavoidable?”

show abstract

Section: Resultssupporting

confidence: 61%

Section: The Uses Of Null Hypothesis Significance Testing Revisited Fmentioning

confidence: 99%

See 1 more Smart Citation

Uses, Abuses and Misuses of Significance Tests in the Scientific Community: Won't the Bayesian Choice Be Unavoidable?

Lecoutre¹,

Lecoutre²,

Poitevineau³

2001

International Statistical Review / Revue Internationale de Stat

View full text Add to dashboard Cite

show abstract

“…Selection is a psychosocial phenomenon in that results are selected, at least in part, based on interpretations of their conclusiveness. Experimental research on the interpretation of statistical results by psychologists (e.g., Rosenthal & Gaito, 1963, 1964Nelson, Rosenthal, & Rosnow, 1986) has shown that people tend to perceive a result to be much more conclusive if its p value is less than one of the conventional levels of significance such as .05 or .01. Thus the use of a discontinuous weight function seems quite reasonable.…”

Section: Selection Modelmentioning

confidence: 98%

Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model

Hedges

Vevea

1996

Journal of Educational and Behavioral Statistics

185

101

View full text Add to dashboard Cite

Keywords: file drawer problem, meta-analysis, publication bias, random effects models, robustness, selection modelsWhen there is publication bias, studies yielding large p values, and hence small effect estimates, are less likely to be published, which leads to biased estimates of effects in meta-analysis. We investigate a selection model based on one-tailed p values in the context of a random effects model. The procedure both models the selection process and corrects for the consequences of selection on estimates of the mean and variance of effect parameters. A test of the statistical significance of selection is also provided. The small sample properties of the method are evaluated by means of simulations, and the asymptotic theory is found to be reasonably accurate under correct model specification and plausible conditions. The method substantially reduces bias due to selection when model specification is correct, but the variance of estimates is increased; thus mean squared error is reduced only when selection produces substantial bias. The robustness of the method to violations of assumptions about the form of the distribution of the random effects is also investigated via simulation, and the model-corrected estimates of the mean effect are generally found to be much less biased than the uncorrected estimates. The significance test for selection bias, however, is found to be highly nonrobust, rejecting at up to 10 times the nominal rate when there is no selection but the distribution of the effects is incorrectly specified.

show abstract

“…For example, replications are successful if both or neither reach p < .05 and they are unsuccessful if one p < .05 and the other p > .05. Psychologists' reliance on a dichotomous decision procedure accompanied by an untenable discontinuity of credibility in results varying in p levels has been well documented {Nelson, Rosenthal, & Rosnow, 1986;Rosenthal & Gaito, 1963. 1964.…”

Section: Contrasting Views Of Replicationmentioning

confidence: 99%

Cumulating Psychology: An Appreciation of Donald T. Campbell

Rosenthal

1991

Psychol Sci

Self Cite

View full text Add to dashboard Cite

Further Evidence for the Cliff Effect in the Interpretation of Levels of Significance

Cited by 57 publications

References 2 publications

Uses, Abuses and Misuses of Significance Tests in the Scientific Community: Won't the Bayesian Choice Be Unavoidable?

Uses, Abuses and Misuses of Significance Tests in the Scientific Community: Won't the Bayesian Choice Be Unavoidable?

Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model

Cumulating Psychology: An Appreciation of Donald T. Campbell

Contact Info

Product

Resources

About