Information theory and hypothesis testing: a call for pluralism

Abstract

Summary 1.A major paradigm shift is occurring in the approach of ecologists to statistical analysis. The use of the traditional approach of null-hypothesis testing has been questioned and an alternative, model selection by information-theoretic methods, has been strongly promoted and is now widely used. For certain types of analysis, information-theoretic approaches offer powerful and compelling advantages over null-hypothesis testing. 2. The benefits of information-theoretic methods are often framed as criticisms of null-hypothesis testing. We argue that many of these criticisms are neither irremediable nor always fair. Many are criticisms of the paradigm's application, rather than of its formulation. Information-theoretic methods are equally vulnerable to many such misuses. Care must be taken in the use of either approach but users of null-hypothesis tests, in particular, must greatly improve standards of reporting and interpretation. 3. Recent critiques have suggested that the distinction between experimental and observational studies defines the limits of the utility of null-hypothesis testing (with the paradigm being applicable to the former but not the latter). However, we believe that there are many situations in which observational data are collected that lend themselves to analysis under the null-hypothesis testing paradigm. We suggest that the applicability of the two analytical paradigms is more accurately defined by studies that assess univariate causality (when null-hypothesis testing is adequate) and those that assess multivariate patterns of causality (when information-theoretic methods are more suitable). 4. Synthesis and applications . Many ecologists are confused about the circumstances under which different inferential paradigms might apply. We address some of the major criticisms of the null-hypothesis testing paradigm, assess those criticisms in relation to the information-theoretic paradigm, propose methods for improving the use of null-hypothesis testing, and discuss situations in which the use of null-hypothesis testing would be appropriate. We urge instructors and practitioners of statistical methods to heighten awareness of the limitations of null-hypothesis testing and to use informationtheoretic methods whenever prior evidence suggests that multiple research hypotheses are plausible. We contend, however, that by marginalizing the use of null-hypothesis testing, ecologists risk rejecting a powerful, informative and well-established analytical tool.

The Value of Applying Commercial Fishers' Experience to Designed Surveys for Identifying Characteristics of Essential Fish Habitat for Adult Summer Flounder

“…Because the interactive approach taken may have led to overfitting of models and overestimates of precision (as reflected in 95% CIs), we considered models obtained from this procedure to be preliminary and subject to further investigation using additional data. However, this approach was consistent with the idea that the survey was designed to identify habitat-CPUE correlations that would require additional experimental work to verify causal relationships (Stephens et al 2005). The effects of water characteristics on summer flounder abundance were tested using single analyses of variance with log e (CPUE þ 1) as the dependent variable and temperature, dissolved oxygen, and salinity as predictors.…”

How safe is mist netting? evaluating the risk of injury and mortality to birds

“…We included a year by organization interaction term because we anticipated that yearly differences that could affect the probability of incident might vary geographically. While the use of null hypothesis testing in observational studies has been criticized (see Burnham & Anderson 2002;Johnson & Omland 2004 among others), it can be appropriate when the primary objective is to determine whether a biologically meaningful difference between groups exists and when only a single hypothesis is being tested (Stephens et al 2005). In GLMM, using LRT for fixed effects is reliable when sample sizes are large relative to the number of parameters (Bolker et al 2008).…”

Calling in sick: impacts of fever on intra-urban human mobility

“…Furthermore, because we performed 12 separate tests for each aspect of mobility, we regarded support for significance between two models as strongest if the p-value of a test was less than a ¼ 0.05/12 ¼ 4.17 Â 10 23 , which was obtained by applying a Bonferroni correction [29]. Although our analysis could have been performed under alternative frameworks, such as information theoretic or Bayesian approaches, our choice of a series of nested likelihood ratio tests was appropriate given our hypothesis testing goals and the nested relationships among our models [30].…”