Do p Values Lose Their Meaning in Exploratory Analyses? It Depends How You Define the Familywise Error Rate

Rubin, Mark A.

doi:10.31234/osf.io/gn8aq

Cited by 24 publications

(26 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that whether p ‐values from exploratory analyses like these can be interpreted is currently being discussed (Nosek, Ebersole, DeHaven, & Mellor, ; Nosek & Lakens, ; Forstmeier, Wagenmakers, & Parker, ; see also Dahl, Grotle, Benth, & Natvig, ; De Groot, ). However, as p ‐values are good predictors of replication success (Open Science Collaboration, ) and recent theorising attests to their validity even in exploratory analyses (Rubin, ), we will use them as guides to highlight certain trends in the data.…”

Section: Resultsmentioning

confidence: 99%

Doing despite Disliking: Self–Regulatory Strategies in Everyday Aversive Activities

2019

View full text Add to dashboard Cite

We investigated the self-regulatory strategies people spontaneously use in their everyday lives to regulate their persistence during aversive activities. In pilot studies (pooled N = 794), we identified self-regulatory strategies from self-reports and generated hypotheses about individual differences in trait self-control predicting their use. Next, deploying ambulatory assessment (N = 264, 1940 reports of aversive/challenging activities), we investigated predictors of the strategies' self-reported use and effectiveness (trait self-control and demand types). The popularity of strategies varied across demands. In addition, people higher in trait self-control were more likely to focus on the positive consequences of a given activity, set goals, and use emotion regulation. Focusing on positive consequences, focusing on negative consequences (of not performing the activity), thinking of the near finish, and emotion regulation increased perceived self-regulatory success across demands, whereas distracting oneself from the aversive activity decreased it. None of these strategies, however, accounted for the beneficial effects of trait self-control on perceived self-regulatory success. Hence, trait self-control and strategy use appear to represent separate routes to good self-regulation. By considering trait-and process-approaches these findings promote a more comprehensive understanding of self-regulatory success and failure during people's daily attempts to regulate their persistence.

show abstract

Section: Resultsmentioning

confidence: 99%

Doing despite Disliking: Self–Regulatory Strategies in Everyday Aversive Activities

2019

View full text Add to dashboard Cite

show abstract

“…All the correlations and ANOVAs were exploratory and performed with an adjusted p-value (p < 0.01). This alpha level was adjusted based on the test of several hypotheses and not on the number of correlations in the exploratory analyses, as suggested by Rubin (2017). Additionally, an ANOVA was performed on the NASA-TLX global score to test one of our main hypotheses (p < 0.01).…”

Section: For the Correlations Between Eye And Cop/body Movementsmentioning

confidence: 99%

Relations between Eye Movement, Postural Sway and Cognitive Involvement in Unprecise and Precise Visual Tasks

et al. 2019

View full text Add to dashboard Cite

Cet article publié par ELSEVIER provient du Dépôt institutionnel de l'Université Libre de Bruxelles, DIfusion http://difusion.ulb.ac.be. Il n'est accessible qu'aux membres de la communauté universitaire de l'ULB sur le réseau sécurisé de l'ULB. Tout utilisateur autorisé peut lire, télécharger ou reproduire cet article à des fins d'usage privé ou à des fins non commerciales d'enseignement ou de recherche scientifique. Il ne peut être atteint à l'intégrité de l'article, et les noms des auteurs et de l'éditeur doivent être conservés. Tout téléchargement systématique des articles publiés par ELSEVIER mis à disposition dans DI-fusion est interdit. This article published by ELSEVIER comes from the Institutional repository of Université Libre de Bruxelles, DI-fusion http://difusion.ulb.ac.be. It is accessible only to the members of the university community of ULB on the ULB secure network. Any authorized user may read, download or reproduce this article for private usage, or for non commercial research or educational purposes. The integrity of the article and identification of the author and copyright owner must be preserved. Systematic downloading of articles published by ELSEVIER that are available in DI-fusion is not permitted. NEUROSCIENCE

show abstract

“…First, we would like to state that we are aware that reporting p-values in exploratory studies is potentially problematic (Rubin, 2017;Neuroskeptic, 2015) because of the open debate of what these p-values really represent (e.g., a null hypothesis significance testing of an absence of relevant factors in the first place). The discussion is so recent that we opted for the conservative choice to use p-values and their classic value for significance (p < .05) as a threshold to include (or exclude) variables in our theory.…”

Section: Discussion Findingsmentioning

confidence: 99%

Peer Review #2 of "A theory on individual characteristics of successful coding challenge solvers (v0.2)"

2019

View full text Add to dashboard Cite

Background. Assessing a software engineer's ability to solve algorithmic programming tasks has been an essential part of technical interviews at some of the most successful technology companies for several years now. We do not know to what extent individual characteristics, such as personality or programming experience, predict the performance in such tasks. Decision makers' unawareness of possible predictor variables has the potential to bias hiring decisions which can result in expensive false negatives as well as in the unintended exclusion of software engineers with actually desirable characteristics. Methods. We conducted an exploratory quantitative study with 32 software engineering students to develop an empirical theory on which individual characteristics predict the performance in solving coding challenges. We developed our theory based on an established taxonomy framework by Gregor (2006). Results. Our findings show that the better coding challenge solvers also have better exam grades and more programming experience. Furthermore, conscientious as well as sad software engineers performed worse in our study. We make the theory available in this paper for empirical testing. Discussion. The theory raises awareness to the influence of individual characteristics on the outcome of technical interviews. Should the theory find empirical support in future studies, hiring costs could be reduced by selecting appropriate criteria for preselecting candidates for on-site interviews and potential bias in hiring decisions could be reduced by taking suitable measures.

show abstract

Do p Values Lose Their Meaning in Exploratory Analyses? It Depends How You Define the Familywise Error Rate

Cited by 24 publications

References 17 publications

Doing despite Disliking: Self–Regulatory Strategies in Everyday Aversive Activities

Doing despite Disliking: Self–Regulatory Strategies in Everyday Aversive Activities

Relations between Eye Movement, Postural Sway and Cognitive Involvement in Unprecise and Precise Visual Tasks

Peer Review #2 of "A theory on individual characteristics of successful coding challenge solvers (v0.2)"

Contact Info

Product

Resources

About