Further Considerations on Testing the Null Hypothesis and the Strategy and Tactics of Investigating Theoretical Models.

Binder, Arnold

doi:10.1037/h0039657

Cited by 72 publications

(38 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One way to increase theoretical precision is to treat the null hypothesis not as a single value but as a range of values that can be considered negligible from a theoretical standpoint (Binder, 1963;Cortina & Dunlap, 1997;Fowler, 1985;Hodges & Lehmann, 1954;Meehl, 1967;Murphy, 1990;Serlin, 1993;Serlin & Lapsley, 1985;Tryon, 2001). With this approach, theories would be required to predict not merely that a parameter differs from zero but that the parameter deviates from zero by some minimum threshold.…”

Section: Expand the Null Hypothesismentioning

confidence: 99%

“…Comparative predictions facilitate the pursuit of strong inference (Platt, 1964) in which rival hypotheses are pitted against one another and studies are designed such that evidence supporting one hypothesis necessarily refutes other hypotheses. Comparative predictions are involved when researchers test competing theories (Binder, 1963;Lakatos, 1978;Meehl, 1990a;Serlin & Lapsley, 1985) and evaluate alternative structural equation models (Anderson & Gerbing, 1988;Rodgers, 2010;Vandenberg & Grelle, 2009). Our focus here is on using comparative predictions to increase the precision of a single theory, such that a theory becomes more precise to the extent its predicted effects can be rank ordered in terms of magnitude.…”

Section: State Predictions As Comparisonsmentioning

confidence: 99%

See 1 more Smart Citation

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Edwards

Berry

2010

Organizational Research Methods

209

210

View full text Add to dashboard Cite

In management research, theory testing confronts a paradox described by Meehl in which designing studies with greater methodological rigor puts theories at less risk of falsification. This paradox exists because most management theories make predictions that are merely directional, such as stating that two variables will be positively or negatively related. As methodological rigor increases, the probability that an estimated effect will differ from zero likewise increases, and the likelihood of finding support for a directional prediction boils down to a coin toss. This paradox can be resolved by developing theories with greater precision, such that their propositions predict something more meaningful than deviations from zero. This article evaluates the precision of theories in management research, offers guidelines for making theories more precise, and discusses ways to overcome barriers to the pursuit of theoretical precision. Keywords philosophy of science, quantitative research, theory developmentTheory testing in management research faces a paradox that confronts many social sciences. This paradox was articulated by Meehl (1967Meehl ( , 1978, who noted that in the hard sciences, such as physics and chemistry, improvements in research design lead to stronger tests of theories, subjecting them to increased risk of falsification. This phenomenon occurs because theories in the hard sciences can produce hypotheses that translate into predictions expressed as point values. As research designs become stronger (e.g., sample sizes are larger, measures have less error), estimates of point values have tighter confidence intervals, which increases the likelihood that hypotheses will be rejected. Hypotheses that survive these increasingly stringent conditions provide stronger corroboration of theories.In the soft sciences, such as management, stronger research designs yield weaker tests of theories. This paradox arises because theories in the soft sciences usually express predictions not as point values, but as directional statements, such as a positive or negative relationship between two variables. These predictions are tested by constructing a confidence interval not around some value predicted by the theory but instead around a null value indicating no effect. As research designs become stronger, confidence intervals around the null value become smaller, and the likelihood of concluding that the estimated effect falls on the predicted side of the null value approaches .50. To illustrate, suppose a theory predicts a positive relationship between two variables, and this prediction is tested with a sample of 380 cases and a p value of .05. Using a conventional two-tailed test, any correlation greater than .10 would be taken as support for the hypothesis. If a one-tailed test were used, taking into account the direction of the hypotheses, only 270 cases would be required to declare a correlation greater than .10 significant at p < .05. Hypotheses that accommodate such a broad range of values confront low hurdle...

show abstract

Section: Expand the Null Hypothesismentioning

confidence: 99%

Section: State Predictions As Comparisonsmentioning

confidence: 99%

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Edwards

Berry

2010

Organizational Research Methods

209

210

View full text Add to dashboard Cite

show abstract

“…Attempts to establish algebraic integration models sometimes cause concern because accepting the model corresponds, in statistical terms, to accepting the null hypothesis in a statistical test (see Binder, 1963;Grant, 1962). In a strict sense, of course, the null hypothesis can never be proved.…”

Section: Accepting the Null Hypothesismentioning

confidence: 99%

Note on functional measurement and data analysis

Anderson

1977

Perception & Psychophysics

View full text Add to dashboard Cite

The guiding idea of functional measurement is that measurement theory and substantive theory form an organic unity. Psychological scales are inherent in the statement of quantitative psychological laws, and these laws themselves form the base and frame for psychological measurement. Valid scales thus depend on empirically valid laws. But establishing empirical validity of any law requires appropriate data analysis. Several statistical problems are discussed with respect to simple algebraic laws. To illustrate the necessity for proper tests of goodness of fit for algebraic models, five sets of experimental data are reanalyzed. In each case, the factorial plot and the analysis of variance showed that the data were nonadditive. Nevertheless, an additive model was fit to the data. The correlations between the data and the predictions from the additive model were extremely high, ranging from .964 to .9997. The corresponding observed-predicted scatterplots also gave little sign of the deviations from additivity. These correlation-scatterplot analyses conceal and obscure what the factorial plot and the analysis of variance reveal and make clear. Other topics discussed are accepting and rejecting the null hypothesis, the use of nonmetric smoothing for parameter estimation, and problems of stimulus-response-model generality.

show abstract

“…And this general finding is considered to be indirect support for Estes & Suppes' treatment of reward magnitude as a special case of reward probability. It is recognized this conclusion implies acceptance of the null hypothesis (see Binder, 1963;Grant, 1962)-but it should be noted that some care was taken initially to have a reasonably large number of experimental subjects. 382 J Fig.…”

Section: Resultsmentioning

confidence: 99%

T- maze learning as a joint function of probability and magnitude of reward

Clayton

Koplin

1964

Psychon Sci

View full text Add to dashboard Cite

Rats were trained on a position discrimination. The experimental design involved simultaneously manipulating per cent and magnitude reward given for the correct response. Learning-rate was found to be a negatively accelerated growth function of reward magnitude for both of the reward schedules used. The results are interpreted as support for the treatment of reward magnitude as a special case of reinforcement probability. ProblemIn a previous study (Clayton, in press) three groups of rats received either 1, 2, or 4 pellets for the correct position response in a T-maze and no reward for an error. Analysis of the resulting performance suggested the following relationship between learning-rate and reward magnitude:where J ~ is the learning-rate parameter associated with r reward pellets. Equation (1) says, generally, that J r is a negatively accelerated growth function of J l' where M is a parameter effecting the growth-rate and asymptote of the curve.The present study was stimulated by an assumption regarding reward magnitude made recently by Estes & Suppes (1959). Following a Guthrian treatment of reward magnitude and distinguishing between reward and reinforcement, Estes & Suppes have suggested reward magnitude is but a special case of rei n for c e men t pro b a b i 1 i t Y in that an increase in magnitude merely increases the probability of the reinforcing event. Within this framework Equation (1) may be conceived as the relationship between learning-rate and reinforcement probability. However, a second variable affecting reinforcement probability should be reward probability, or reward schedule. In the previous study the correct response was always rewarded. The present study sought to determine the effect on the relationship between learning-rate and reward magnitude when the correct response was sometimes nonrewarded. The hypothesis tested was that if Equation (1) relates learning-rate to reinforcement probability, it should hold (with possible changes in M) regardless of whether the correct response was always rewarded.A second purpose of this study was to provide an independent check on Equation (1) with different reward magnitudes to make certain the obtained function is not specific to the particular amounts used previously. For these reasons reward magnitude was factorially manipu-

show abstract

Further Considerations on Testing the Null Hypothesis and the Strategy and Tactics of Investigating Theoretical Models.

Cited by 72 publications

References 34 publications

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research

Note on functional measurement and data analysis

T- maze learning as a joint function of probability and magnitude of reward

Contact Info

Product

Resources

About