The Johnson-Neyman Procedure as an Alternative to ANCOVA

D’Alonzo, Karen T.

doi:10.1177/0193945904266733

Cited by 73 publications

(36 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These statements give a much more detailed and substantial description than for instance 'the effect increased with increasing value of the covariate'. The use of the Johnson-Neyman technique is hampered by its absence as a standard feature in most commercially available statistical packages (but see Hunka & Leighton 1997;D'Alonzo 2004). However, the calculations for the simple, but very common, one covariate-one category with two groups-case are not so demanding and can be obtained in for instance Huitema (1980), White (2003) and D'Alonzo (2004).…”

Section: Discussionmentioning

confidence: 99%

The mistreatment of covariate interaction terms in linear model analyses of behavioural and evolutionary ecology studies

2005

View full text Add to dashboard Cite

In behavioural and evolutionary ecology, there are often large phenotypic differences between individuals in, for example, body size or large variation in abiotic conditions such as temperature, between measurements. This often inevitable source of variation may mask any effect of experimental treatment as it can have a large impact on the dependent variable of interest. In such cases, conventional statistical comparisons may have much lower power than desired. The inclusion of covariates in statistical analyses has proven a powerful method to control for such nonrandom differences between individual data points that cannot be controlled experimentally (Huitema 1980). To make correct conclusions, it is important to understand the basic assumptions underlying such a covariate analysis. In this paper I argue that this has evidently not been completely acknowledged in the scientific community. Sophisticated models relating responses to both one or more continuous covariates and one or more factors can be problematic. Factor is here used in the meaning of a categorical independent variable and its value divides individuals into discrete groups or categories, for instance experimental treatments. In the following, I use a simple one-factor ANCOVA design as an example, but the same general problem outlined here applies to all linear models with one or more covariates, including generalized linear models (GLIM) such as logistic regressions and even survival analysis.The basic design of a one-factor fixed effect ANCOVA can be written as:where Y ij denotes the values for the dependent response variable of the jth subject in the ith category of the factor, m is the mean intercept (the average value of the response parameter when the value of the covariate equals zero), a i is the response to the ith category of the factor, X ij the value for the covariate of the jth subject in the ith category of the factor, X is the mean value of the covariate for all individuals, b is the overall pooled regression coefficient (slope) within groups and e the normally distributed error variance (cf. Huitema 1980). When performing an ANCOVA, we thus assume equivalent slopes among treatment groups (b). The test of homogeneity among slopes is therefore a key prerequisite to proceed to the ANCOVA itself. The easiest way to test this assumption is to include the interaction term between the covariate and the factor in the model. If the interaction term is nonsignificant, we can conclude that the slopes are homogeneous and then proceed to test whether the response differs between groups. This is formally done by testing for differences between treatment groups in the Y intercept for the regressions of the covariate on the response variable Y. This test is carried out by re-running the model, excluding the interaction term. Differences in the Y intercept (i.e. differences between groups when the value for the covariate equals zero) are generally of minor importance. However, since we assume homogeneity of slopes this difference can be inferred o...

show abstract

Section: Discussionmentioning

confidence: 99%

The mistreatment of covariate interaction terms in linear model analyses of behavioural and evolutionary ecology studies

2005

View full text Add to dashboard Cite

show abstract

“…Accordingly, an alternative approach to ANCOVA must be sought. Formulae provided by Potthoff (1964) as a modification to the Johnson-Neyman procedure (Aiken & West, 1991;D'Alonzo, 2004;Pedhazur & Schmelkin, 1991;Rogosa, 1981) is one straightforward alternative.…”

Section: Resultsmentioning

confidence: 99%

“…Consequently, alternative analytical approaches must be considered, which may include the Johnson-Neyman procedure (Fraas & Newman, 1997;Karpman, 1983;Kowalski, Schneiderman, & Willis, 1994;Rogosa, 1981) and extensions thereof, such as that proposed by Potthoff (1964). For the sake of simplicity, let's consider formulae constructed by Potthoff (1964) as a modification to the Johnson-Neyman procedure (Aiken & West, 1991;D'Alonzo, 2004;Pedhazur & Schmelkin, 1991;Rogosa, 1981). These formulae allow for calculations of the point of intersection (crossover point) of regression lines, and what are known as simultaneous regions of significance (SROS).…”

Section: Identify An Alternative Analytical Approachmentioning

confidence: 99%

Violation of the homogeneity of regression slopes assumption in ANCOVA for two-group pre-post designs: Tutorial on a modified Johnson-Neyman procedure

Johnson¹

2016

TQMP

View full text Add to dashboard Cite

Aptitude-treatment interaction (ATI) effects are present when individuals demonstrate differential outcomes across treatments based upon aptitude-that is, any measurable individual characteristic, attribute, or ability (e.g., anxiety, learning style, motivation, prior knowledge). ATI effects may exist in data from one design commonly used in psychological and educational research-the two-group pre-post design-in which pre-intervention scores may be considered to reflect individual aptitude. Researchers may mistakenly overlook these effects, however, due to inappropriate analytical approaches. When applying analysis of covariance (ANCOVA), it is important to check for ANCOVA assumptions, including an assumption known as homogeneity of regression slopes. When heterogeneity of regression slopes is found, ATI effects are revealed. Consequently, alternative approaches to ANCOVA must be sought. Using formulae based on the Johnson-Neyman procedure to define simultaneous regions of significance is one straightforward alternative. This tutorial outlines the process for analyzing data resulting from two-group pre-post studies when data violate the ANCOVA assumption of homogeneity of regression slopes. What was initially viewed as an obstacle may result in the discovery of an ATI effect, which may be described statistically through simple mathematical calculations.

show abstract

“…Accordingly, simultaneous regions of significance were calculated to identify the pre-test score ranges for which social learning contexts differed significantly on post-test scores. Calculations were based upon formulae constructed by Potthoff (1964) as a modification to the Johnson–Neyman approach (Aiken & West 1991; Pedhazur & Schmelkin 1991; D’Alonzo 2004). Follow-up independent samples t -tests were conducted to further describe the inferences of these calculations for a sub-set of the sample.…”

Section: Methodsmentioning

confidence: 99%

Virtual patient simulations and optimal social learning context: A replication of an aptitude–treatment interaction effect

et al. 2014

View full text Add to dashboard Cite

Background Virtual patients (VPs) offer valuable alternative encounters when live patients with rare conditions, such as cranial nerve (CN) palsies, are unavailable; however, little is known regarding simulation and optimal social learning context. Aim Compare learning outcomes and perspectives between students interacting with VPs in individual and team contexts. Methods Seventy-eight medical students were randomly assigned to interview and examine four VPs with possible CN damage either as individuals or in three-person teams, using Neurological Examination Rehearsal Virtual Environment (NERVE). Learning was measured through diagnosis accuracy and pre-/post-simulation knowledge scores. Perspectives of learning context were collected post-simulation. Results Students in teams submitted correct diagnoses significantly more often than students as individuals for CN-IV (p = 0.04; team = 86.1%; individual = 65.9%) and CN-VI (p = 0.03; team = 97.2%; individual = 80.5%). Knowledge scores increased significantly in both contexts (p<50.001); however, a significant aptitude–treatment interaction effect was observed (p = 0.04). At pre-test scores ≤25.8%, students in teams scored significantly higher (66.7%) than students as individuals (43.1%) at post-test (p = 0.03). Students recommended implementing future NERVE exercises in teams over five other modality-timing combinations. Conclusion Results allow us to define best practices for integrating VP simulators into medical education. Implementing NERVE experiences in team environments with medical students in the future may be preferable.

show abstract

The Johnson-Neyman Procedure as an Alternative to ANCOVA

Cited by 73 publications

References 3 publications

The mistreatment of covariate interaction terms in linear model analyses of behavioural and evolutionary ecology studies

The mistreatment of covariate interaction terms in linear model analyses of behavioural and evolutionary ecology studies

Violation of the homogeneity of regression slopes assumption in ANCOVA for two-group pre-post designs: Tutorial on a modified Johnson-Neyman procedure

Virtual patient simulations and optimal social learning context: A replication of an aptitude–treatment interaction effect

Contact Info

Product

Resources

About