If the null hypothesis is impossible, why test it?

Murphy, Kevin R.

doi:10.1037/0003-066x.45.3.403

Cited by 25 publications

(15 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One way to increase theoretical precision is to treat the null hypothesis not as a single value but as a range of values that can be considered negligible from a theoretical standpoint (Binder, 1963;Cortina & Dunlap, 1997;Fowler, 1985;Hodges & Lehmann, 1954;Meehl, 1967;Murphy, 1990;Serlin, 1993;Serlin & Lapsley, 1985;Tryon, 2001). With this approach, theories would be required to predict not merely that a parameter differs from zero but that the parameter deviates from zero by some minimum threshold.…”

Section: Expand the Null Hypothesismentioning

confidence: 99%

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Edwards

Berry

2010

Organizational Research Methods

209

210

View full text Add to dashboard Cite

In management research, theory testing confronts a paradox described by Meehl in which designing studies with greater methodological rigor puts theories at less risk of falsification. This paradox exists because most management theories make predictions that are merely directional, such as stating that two variables will be positively or negatively related. As methodological rigor increases, the probability that an estimated effect will differ from zero likewise increases, and the likelihood of finding support for a directional prediction boils down to a coin toss. This paradox can be resolved by developing theories with greater precision, such that their propositions predict something more meaningful than deviations from zero. This article evaluates the precision of theories in management research, offers guidelines for making theories more precise, and discusses ways to overcome barriers to the pursuit of theoretical precision. Keywords philosophy of science, quantitative research, theory developmentTheory testing in management research faces a paradox that confronts many social sciences. This paradox was articulated by Meehl (1967Meehl ( , 1978, who noted that in the hard sciences, such as physics and chemistry, improvements in research design lead to stronger tests of theories, subjecting them to increased risk of falsification. This phenomenon occurs because theories in the hard sciences can produce hypotheses that translate into predictions expressed as point values. As research designs become stronger (e.g., sample sizes are larger, measures have less error), estimates of point values have tighter confidence intervals, which increases the likelihood that hypotheses will be rejected. Hypotheses that survive these increasingly stringent conditions provide stronger corroboration of theories.In the soft sciences, such as management, stronger research designs yield weaker tests of theories. This paradox arises because theories in the soft sciences usually express predictions not as point values, but as directional statements, such as a positive or negative relationship between two variables. These predictions are tested by constructing a confidence interval not around some value predicted by the theory but instead around a null value indicating no effect. As research designs become stronger, confidence intervals around the null value become smaller, and the likelihood of concluding that the estimated effect falls on the predicted side of the null value approaches .50. To illustrate, suppose a theory predicts a positive relationship between two variables, and this prediction is tested with a sample of 380 cases and a p value of .05. Using a conventional two-tailed test, any correlation greater than .10 would be taken as support for the hypothesis. If a one-tailed test were used, taking into account the direction of the hypotheses, only 270 cases would be required to declare a correlation greater than .10 significant at p < .05. Hypotheses that accommodate such a broad range of values confront low hurdle...

show abstract

Section: Expand the Null Hypothesismentioning

confidence: 99%

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Edwards

Berry

2010

Organizational Research Methods

209

210

View full text Add to dashboard Cite

show abstract

“…Thus, the hypothesis of "no synchrony exists between two individuals" is theoretically impossible and can easily be falsely rejected by standard null-hypothesis testing methods. This problem of testing impossible null-hypotheses is a common critique of standard null-hypothesis significance testing as it promotes the occurrence of type-I error (Cohen, 1994; Murphy, 1990; Szucs & Ioannidis, 2017). Indeed, standard null-hypothesis testing methods have been recognized as having multiple problems, from misunderstandings of p-values to what some researchers see as an over-reliance on this method for progressing scientific theory; see Harlow, Mulaik, and Steiger (2016) for an overview of these problems and counterarguments.…”

Section: Synchrony Vs Pseudosynchronymentioning

confidence: 99%

Determining synchrony between behavioral time series: An application of surrogate data generation for establishing falsifiable null-hypotheses.

Moulder¹,

Boker²,

Ramseyer³

et al. 2018

Psychological Methods

View full text Add to dashboard Cite

Synchrony between interacting systems is an important area of nonlinear dynamics in physical systems. Recently psychological researchers from multiple areas of psychology have become interested in nonverbal synchrony (i.e., coordinated motion between two individuals engaged in dyadic information exchange such as communication or dance) as a predictor and outcome of psychological processes. An important step in studying nonverbal synchrony is systematically and validly differentiating synchronous systems from nonsynchronous systems. However, many current methods of testing and quantifying nonverbal synchrony will show some level of observed synchrony even when research participants have not interacted with one another. In this article we demonstrate the use of surrogate data generation methodology as a means of testing new null-hypotheses for synchrony between bivariate time series such as those derived from modern motion tracking methods. Hypotheses generated by surrogate data generation methods are more nuanced and meaningful than hypotheses from standard null-hypothesis testing. We review four surrogate data generation methods for testing for significant nonverbal synchrony within a windowed cross-correlation (WCC) framework. We also interpret the null-hypotheses generated by these surrogate data generation methods with respect to nonverbal synchrony as a specific use of surrogate data generation, which can then be generalized for hypothesis testing of other psychological time series. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

show abstract

“…It is virtually impossible to design real treatments or interventions that have no effect whatsoever (Murphy, 1990;Murphy and Myors, 1998); treatment effects might be trivially small, but they are rarely exactly nil (Cohen, 1994). Power therefore can usually be thought of as the probability that your study will confirm what you already knowi.e., that treatments probably have some effect, although these effects might be so small that they are meaningless (Murphy and Myors, 1999).…”

Section: Why You Should Care About Powermentioning

confidence: 99%

Using Power Analysis to Evaluate and Improve Research

Murphy¹

2004

Handbook of Research Methods in Industrial and Organizational Psychology

View full text Add to dashboard Cite

One of the most common statistical procedures in the behavioral and social sciences is to test the hypothesis that treatments or interventions have no effect, or that the correlation between two variables is equal to zero, etc. Null hypothesis (H 0 ) tests have long been viewed as a critical part of the research process, and in the mind of some researchers, statistical analyses start and end with these "significance tests." Power analyses deal with the relationships between the structure of these statistical tests, the nature of the phenomenon that is being studied, and the likelihood that these tests will consistently detect the effects of treatments, interventions, and the like (Cohen, 1994;Murphy and Myors, 1998). There is a substantial controversy about the value and meaning of null hypothesis

show abstract

If the null hypothesis is impossible, why test it?

Cited by 25 publications

References 9 publications

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Determining synchrony between behavioral time series: An application of surrogate data generation for establishing falsifiable null-hypotheses.

Using Power Analysis to Evaluate and Improve Research

Contact Info

Product

Resources

About