Assessing the Evidence in Evidence‐Based Medicine

2020

BMC Med Res Methodol

Background: There is a growing trend in using the "statistically significant" term in the scientific literature. However, harsh criticism of this concept motivated the recommendation to withdraw its use of scientific publications. We aimed to validate the support and the feasibility of adherence to this recommendation, among researchers having declared in favor of removing the statistical significance. Methods: We surveyed signatories of an article published that defended this recommendation, to validate their opinion and ask them about how likely they will retire the concept of statistical significance. Results: We obtained 151 responses which confirmed the support for the mentioned publication in aspects such as the adequate interpretation of the p-value, the degree of agreement, and the motivations to sign it. However, there was a wide distribution of answers about how likely are they to use the concept of "statistical significance" in future publications. About 42% declared being neutral, or that would likely use it again. We described arguments referred by several signatories and discussed aspects to be considered in the interpretation of research results. Conclusions: The responses obtained from a proportion of signatories validated their declared position against the use of statistical significance. However, even in this group, the full application of this recommendation does not seem feasible. The arguments related to the inappropriate use of statistical tests should promote more education among researchers and users of scientific evidence.

Section: Discussionmentioning

confidence: 99%

How feasible is it to abandon statistical significance? A reflection based on a short survey

2020

BMC Med Res Methodol

“…In those cases, the efforts must focus on adequately research designing and analysis to avoid bias, control confusion, and consider eventual effect modifications. [18,19]. After that, the measures of association and impact should define when a result is significant in the clinical and public health scopes [20].…”

Section: Discussionmentioning

confidence: 99%

How feasible is it to abandon statistical significance? A reflection based on a short survey

2020

Preprint

Background: There is a growing trend in using the “statistically significant” term in the scientific literature. However, harsh criticism of this concept motivated the recommendation to withdraw its use of scientific publications. We aimed to validate the support and the feasibility of adherence to this recommendation, among researchers having declared in favor of removing the statistical significance. Methods: We surveyed signatories of an article published that defended this recommendation, to validate their opinion and ask them about how likely they will retire the concept of statistical significance. Results: We obtained 151 responses which confirmed the support for the mentioned publication in aspects such as the adequate interpretation of the p-value, the degree of agreement, and the motivations to sign it. However, there was a wide distribution of answers about how likely are they to use the concept of "statistical significance” in future publications. About 42% declared being neutral, or that would likely use it again. We described arguments referred by several signatories and discussed aspects to be considered in the interpretation of research results.Conclusions: The responses obtained from a proportion of signatories validated their declared position against the use of statistical significance. However, even in this group, the full application of this recommendation does not seem feasible. The arguments related to the inappropriate use of statistical tests should promote more education among researchers and users of scientific evidence.

“…However, in studies of causal inference, the concept of statistical significance should not be a primary concern. Before looking at a p-value, the researcher will have to avoid biases and look on conceptual structures to control confusion and consider contexts in which effects can be modified [18,19]. After that, the measures of association and impact are those that must define when a result is significant in the clinical and public health scopes [20].…”

Section: Discussionmentioning

confidence: 99%

How feasible is it to abandon statistical significance? A reflection based on a short survey

2019

Preprint

Background : There is a growing trend in using the “statistically significant” term in the scientific literature. However, harsh criticism of this concept motivated the recommendation to withdraw its use of scientific publications. We aimed to validate the support and the feasibility of adherence to this recommendation, among researchers having declared in favor of removing the statistical significance. Methods : We surveyed signatories of an article published that defended this recommendation, to validate their opinion and ask them about how likely they will retire the concept of statistical significance. Results: We obtained 151 responses which confirmed the support for the mentioned publication in aspects such as the adequate interpretation of the p-value, the degree of agreement, and the motivations to sign it. However, there was a wide distribution of answers about how likely are they to use the concept of "statistical significance” in future publications. About 42% declared being neutral, or that would likely use it again. We described arguments referred by several signatories and discussed aspects to be considered in the interpretation of research results. Conclusions : The declared position against the use of statistical significance had had legitimate support from numerous researchers. However, the full application of this recommendation does not seem feasible. The arguments related to the inappropriate use of statistical tests should promote more education among researchers and users of scientific evidence.