Linear algebra and multivariate analysis in statistics: development and interconnections in the twentieth century

Abstract

The most obvious points of contact between linear and matrix algebra and statistics are in the area of multivariate analysis. We review the way that, as both developed during the last century, the two influenced each other. We illustrate this by examining a number of key areas.We begin with matrix and linear algebra, its emergence in the 19th century, and its eventual penetration into the undergraduate curriculum in the 20th century. We continue with a much longer similar account for multivariate analysis in statistics. We pick out the year 1936 as that of three key developments, by H. Hotelling, R. A. Fisher and P. C. Mahalanobis, and in the early post-war period three more key developments, by M. G. Kendall, M. S. Bartlett and C. R. Rao. We then turn to some special results in linear algebra that we need: Schur complements, and related inversion formulae. We briefly discuss four of the contributors, Fisher and Kendall named above, together with S. S. Wilks and T. W. Anderson. We close with thirteen 'case studies', showing in a range of specific cases how these general algebraic methods have been put to good use and changed the face of statistics.Keywords linear algebra, matrix algebra, multivariate analysis in statistics, multivariate normal distribution, Gaussian Regression Formula, Schur complement, singular values decomposition. Time-lineOur main focus is on the 20th century, though we touch on the 18th and 19th (and of course are influenced by the first part of the 21st). Both authors were born in the 1940s and began publishing in the 1970s. We allow ourselves to use the term 'modern', imprecise and subjective as it is, as being as useful in this context as in general usage. It might be loosely read here as 'within living memory'. For example, we say ( §4.3) of Anderson's 1958 book that we regard it as 'the first of the unambiguously modern textbooks we cite'. We invite readers to form their own views here.see Eisenhart (1961)) helped to motivate the development of matrix algebra and linear programming. A study of the emergence of matrix theory as a subject in its own right, with particular reference to the inter-war years, is given by Brechenmacher (2010).J. W. Gibbs, in his lecture notes at Yale of 1881 and 1884 (published 1901), did much to spread the use of vector methods. An early stimulus for these was Grassmann's Ausdehnungslehre of 1844. Vector methods were championed in the UK by O. Heaviside (to whom we owe, e.g., the use of bold face for vectors). For a good brief account of the struggles between the proponents of vectors and quaternions (W. R. Hamilton, 1843), see the Historical Introduction in (Weatherburn 1921). In a paper written while he was still a student, Fisher applied vector methods to geometry (Fisher 1913). Unfortunately, vector methods were still scorned by some as late as 1937; see e.g. the Preface of (Ramsey, 1937).An insight into the our theme a century earlier is provided by the following passage from (Todhunter 1869) acknowledging input from Cayley (thanks to Steve Stigler for t...

Algebraic Treatments of a General Linear Model With Adding-up Restrictions

“…The theory of generalized inverses of matrices is a main source of the techniques that were brought into linear regression analysis in 1950s, and thus plays a key role for carrying out the estimation and inference in a wide variety of situations; see e.g., Bingham & Krzanowski (2022); Puntanen et al (2011); Searle (1982). In order to simplify various matrix expressions involving generalized inverses of matrices, we need to use the following rank formulas.…”

Mengeksplore aljabar awal di sekolah menengah: kombinasi analisis bibliometric dan meta analisis

“…Algebraic concepts emerged during the 19th century, and their eventual integration into undergraduate curricula took place in the 20th century ( Stout et al, 1871;Mumford et al, 1981;Feferman et al, 2002;Macdonald et al, 2011;Stout, 2021;Bingham & Krzanowski, 2022). However, ongoing discussions in academic literature increasingly emphasize the drawbacks of separating the teaching of arithmetic and early algebra (Carraher et al, 2006;Warren, 2004;Welder, 2012).…”