Lord’s Paradox Revisited – (Oh Lord! Kumbaya!)

Abstract

Among the many peculiarities that were dubbed "paradoxes" by well meaning statisticians, the one reported by Frederic M. Lord in 1967 has earned a special status. Although it can be viewed, formally, as a version of Simpson's paradox, its reputation has gone much worse. Unlike Simpson's reversal, Lord's is easier to state, harder to disentangle and, for some reason, it has been lingering for almost four decades, under several interpretations and re-interpretations, and it keeps coming up in new situations and under new lights. Most peculiar yet, while some of its variants have received a satisfactory resolution, the original version presented by Lord, to the best of my knowledge, has not been given a proper treatment, not to mention a resolution.The purpose of this paper is to trace back Lord's paradox from its original formulation, resolve it using modern tools of causal analysis, explain why it resisted prior attempts at resolution and, finally, address the general methodological issue of whether adjustments for preexisting conditions is justified in group comparison applications.Keywords: baseline adjustment, birth-weight paradox, differential base-rate, group comparison, Simpson's paradox, sure-thing principle 1 Lord's original dilemma Any attempt to describe Lord's paradox in words other than those used by Lord himself can only do injustice to the clarity and freshness with which it was first enunciated in 1967 [1]. We will begin therefore by listening to Lord's own words."A large university is interested in investigating the effects on the students of the diet provided in the university dining halls and any sex difference in these effects. Various types of data are gathered. In particular, the weight of each student at the time of his arrival in September and his weight the following June are recorded.At the end of the school year, the data are independently examined by two statisticians. Both statisticians divide the students according to sex. The first statistician examines the mean weight of the girls at the beginning of the year and at the end of the year and finds these to be identical. On further investigation, he finds that the frequency distribution of weight for the girls at the end of the year is actually the same as it was at the beginning.He finds the same to be true for the boys. Although the weight of individual boys and girls has usually changed during the course of the year, perhaps by a considerable amount, the group of girls considered as a whole has not changed in weight, nor has the group of boys. A sort of dynamic equilibrium has been maintained during the year.The whole situation is shown by the solid lines in the diagram (Figure 1). Here the two ellipses represent separate scatterplots for the boys and the girls. The frequency distributions of initial weight are indicated at the top of the diagram and the identical distributions of final weight are indicated on the left side. People falling on the solid 45°line through the origin are people whose initial and final weight ar...

Allocation to groups: Examples of Lord's paradox

“…4) perform better than the other. The results of Holland & Rubin (1983), Pearl (2016), and Wright (2006) reinforce the notion that: "knowledge of the assignment process is critical to drawing inferences about the effect of the treatments" (Rubin, 1977, p. 22). A simulation will be conducted to show how the way in which students are assigned to schools should affect the choice of statistical method.…”

“…21-22) describe this situation in §A.4 of their appendix and show (assuming linearity and parallel slopes for the groups) that the ANCOVA approach yields appropriate estimates. Pearl (2016) uses graphical models and reaches a similar conclusion: in Panel A both approaches can be correct depending on the research question and assumptions, but for Panel B "one [statistician] was right (ANCOVA) and one [statistician] was wrong." Wright (2006) reached similar conclusions, but using simulation methods.…”

“…Several authors (e.g., Holland & Rubin, 1983;Pearl, 2016;Wright, 2006) discuss that it is important to consider whether the PreTest scores influence the grouping variable. These authors use of mathematical, graphical, and simulation methods, and their texts will appear quite technical to many readers.…”

The effect of the “Follow in my Green Food Steps” programme on cooking behaviours for improved iron intake: a quasi-experimental randomized community study

“…The two towns had baseline differences, which an analysis of covariance (ANCOVA) does not sufficiently attenuate for [ 66 – 69 ]. Analyses were therefore conducted using both change scores (non-parametric Wilcoxon signed-rank tests) and ANCOVAs.…”

Explaining Lord's Paradox in Introductory Statistical Theory Courses

“…The paradox has been studied in the context of health sciences, environmental sciences, and psychometrics (Holland & Rubin, 1983;Wainer & Brown, 2006;Glymour et al, 2005;Tu et al, 2008;Pearl, 2016). It is an interesting phenomenon which frequently occurs in practice, but it is not easy to quantify the exact difference between the parameter in the two-sample t-test and the parameter in the ANCOVA model without statistical theory.…”