Testing Significance Testing

Krueger, Joachim I.; Heck, Patrick R.

doi:10.1525/collabra.108

Cited by 8 publications

(7 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used non-parametric tests for the hypothesis tests when assumptions for parametric tests were not met, and, where appropriate, performed Bayesian hypothesis evaluation. All parametric, nonparametric, and Bayesian analyses supported the same substantive conclusions, a congruence we anticipated (Krueger & Heck, 2018).…”

Section: Resultssupporting

confidence: 68%

Hard Feelings? Predicting Attitudes Toward Former Romantic Partners

Grüning¹,

Loose²,

Krueger³

2021

Preprint

View full text Add to dashboard Cite

Most research on relationship quality addresses ongoing involvements. Research on past relationships is rare. As a first step, Athenstaedt and colleagues (2020) explored attitudes toward former romantic partners in an Austrian sample of heterosexual respondents. They found that men held less negative attitudes than women. In two studies conducted in Germany and the USA, we replicate this gender difference and explore the role of three psychological predictors. Like Athenstaedt et al., we find that the degree of perceived social support before the breakup and continued friendly relations after the breakup have a positive association with ex-partner attitude. Critically, we introduce and corroborate the hypothesis that regret over having started the relationship has a negative association. However, regret also fails to mediate the association between gender and ex-partner attitude. We discuss the practical implications of these findings and note directions for future research.

show abstract

Section: Resultssupporting

confidence: 68%

Hard Feelings? Predicting Attitudes Toward Former Romantic Partners

Grüning¹,

Loose²,

Krueger³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Both p values and Bayes factors can be calculated from the t-statistic and the sample size, so it is expected that they would be related. In these simulations, there was a nearperfect linear relationship between the (log of the) Bayes factors and the (log of the) p values, as has been shown previously (Benjamin et al, 2017;Krueger & Heck, 2018;Wetzels et al, 2011). Equivalency in AUCs between Bayes factors and p values generalized to other scenarios as well including one-sample t-tests and correlations (see Figure 8).…”

Section: Bayes Factor >3supporting

confidence: 79%

“…Put another way, the discriminability of p values and Bayes factors are high in situations for which real effects are likely and in situations for which real effects are unlikely. Obviously, more p values and Bayes factors reach thresholds for significance when there are more significant effects, so "significant" effects are more for 'safe' studies than 'risky' studies (Krueger & Heck, 2018). Nevertheless, the diagnosticity of the p value (and of Bayes factor) is high regardless of the likelihood of finding a real effect.…”

Section: Equationmentioning

confidence: 99%

Insights into Criteria for Statistical Significance from Signal Detection Analysis

Witt

2019

View full text Add to dashboard Cite

What is best criterion for determining statistical significance? In psychology, the criterion has been p < .05. This criterion has been criticized since its inception, and the criticisms have been rejuvenated with recent failures to replicate studies published in top psychology journals. Several replacement criteria have been suggested including reducing the alpha level to .005 or switching to other types of criteria such as Bayes factors or effect sizes. Here, various decision criteria for statistical significance were evaluated using signal detection analysis on the outcomes of simulated data. The signal detection measure of area under the curve (AUC) is a measure of discriminability with a value of 1 indicating perfect discriminability and 0.5 indicating chance performance. Applied to criteria for statistical significance, it provides an estimate of the decision criterion’s performance in discriminating real effects from null effects. AUCs were high (M = .96, median = .97) for p values, suggesting merit in using p values to discriminate significant effects. AUCs can be used to assess methodological questions such as how much improvement will be gained with increased sample size, how much discriminability will be lost with questionable research practices, and whether it is better to run a single high-powered study or a study plus a replication at lower powers. AUCs were also used to compare performance across p values, Bayes factors, and effect size (Cohen’s d). AUCs were equivalent for p values and Bayes factors and were slightly higher for effect size. Signal detection analysis provides separate measures of discriminability and bias. With respect to bias, the specific thresholds that produced maximally-optimal utility depended on sample size, although this dependency was particularly notable for p values and less so for Bayes factors. The application of signal detection theory to the issue of statistical significance highlights the need to focus on both false alarms and misses, rather than false alarms alone.

show abstract

“…We used two approaches to hypothesis testing (Primoceri et al, 2021) with the general expectation that they would yield converging conclusions (Krueger & Heck, 2018).…”

Section: Resultsmentioning

confidence: 99%

Strategic thinking in the shadow of self-enhancement: Benefits and costs

Grüning¹,

Krueger²

2022

Preprint

View full text Add to dashboard Cite

Using a variant of the hide-and-seek game, we show in two studies that self-enhancement can help or hinder strategic thinking. In the game, players either choose a number or attempt to guess that number. They can do so by randomizing or by active thinking. Guessers profit from thinking of a number, whereas choosers suffer from it. Yet, irrespective of their role, respondents prefer to actively think of a number when the opponent thinks. Participants believe they can outthink an opponent and most even prefer to play the game in the chooser’s role, thus neglecting the guesser’s statistical advantage. These findings suggest that the impacts of self-enhancement on strategic thinking are robust, presenting both risks (for the chooser) and opportunities (for the guesser). We discuss the implications of these findings for theories of strategic cognition and applications to real-life contexts.

show abstract

Testing Significance Testing

Cited by 8 publications

References 65 publications

Hard Feelings? Predicting Attitudes Toward Former Romantic Partners

Hard Feelings? Predicting Attitudes Toward Former Romantic Partners

Insights into Criteria for Statistical Significance from Signal Detection Analysis

Strategic thinking in the shadow of self-enhancement: Benefits and costs

Contact Info

Product

Resources

About