Subliminal or not? Comparing null-hypothesis and Bayesian methods for testing subliminal priming

Abstract

A difficulty for reports of subliminal priming is demonstrating that participants who actually perceived the prime are not driving the priming effects. There are two conventional methods for testing this. One is to test whether a direct measure of stimulus perception is not significantly above chance on a group level. The other is to use regression to test if an indirect measure of stimulus processing is significantly above zero when the direct measure is at chance. Here we simulated samples in which we assumed that only participants who perceived the primes were primed by it. Conventional analyses applied to these samples had a very large error rate of falsely supporting subliminal priming. Calculating a Bayes factor for the samples very seldom falsely supported subliminal priming. We conclude that conventional tests are not reliable diagnostics of subliminal priming. Instead, we recommend that experimenters calculate a Bayes factor when investigating subliminal priming.

The importance of awareness for understanding language.

“…The van Gaal (2014) work is also subject to a recently raised methodological criticism (Sand & Nilsson, 2016;Vadillo, Konstantinidis, & Shanks, 2016), that the study's explicit measure of awareness lacked statistical power compared to the main manipulation (80 trials in the awareness test, more than 1000 trials in the main experiment), implying that some participants were likely to have been aware of the stimuli, but failed to statistically demonstrate this awareness in the awareness test. A similar point can be raised about Armstrong and Dienes (2013), who also examined the effect of subliminal negation, and also used a low power procedure to measure individual masking thresholds.…”

Knowledge about the predictive value of reward conditioned stimuli modulates their interference with cognitive processes

“…This typically involves asserting the null hypothesis that performance on some knowledge-related task is no different to chance, which cannot be evaluated using traditional frequentist statistical analyses (Dienes 2015). Therefore, a Bayesian approach (Sand and Nilsson 2016) was used to categorize participants as Aware or Unaware of contingencies using the data gathered from the conditioning procedure (Leganes-Fonteneau et al 2018).…”

“…In a recent series of experiments, in which CA was carefully measured using a novel Bayesian approach (Dienes 2015;Sand and Nilsson 2016), we found that targets, paired with increased probabilities of monetary reward, gathered preferential attention in an Emotional Attentional Blink task (Leganes-Fonteneau et al 2018). Importantly, this was observed in participants Unaware of stimulus-reward contingencies.…”

The Cue-integrated Sense of Agency – as Operationalized in Experiments – is not a (Multisensory) Perceptual Effect, but is a Judgment Effect

“…In this spirit, some researchers argued that priming studies are confounded by methodological problems (Colzato, Warrens, and Hommel, 2006;Kahneman, 2012;Molden, 2014aMolden, , 2014bSand and Nilsson, 2016) and there are replication failures in the domain of priming studies (Doyen et al, 2012;Harris et al, 2013;Shanks et al, 2013;Cesario and Jonas, 2014;Dijksterhuis, Van Knippenberg, and Holland, 2014;Joseph Cesario, 2014;Barry Schwartz, 2015;Daniel Lakens, 2017;Schimmack, Heene, and Kesavan, 2017;O'Donnell et al, 2018). The spreading of activation and associative concept use as well as replication failures of priming studies can be connected to the tendency of compositionality, particularly productivity and systematicity of concepts/thoughts.…”