Orthogonal and oblique projectors and the characteristics of pairs of vector spaces

Abstract

The statistical operation of multiple linear regression by least squares is equivalent to the orthogonal projection of vectors of observations on a space spanned by vectors of observations; and a partial regression can be similarly represented as an oblique projection. This connexion between a statistical and a formal algebraical operation gives the main source of interest for this investigation. Its object is to develop algebraical theory which supplies terms necessary for a unified algebraical and geometrical formulation of concepts in multivariate analysis. An exposition of the statistical theory is to be made in a separate paper.

The many proofs of an identity on the norm of oblique projections

“…Some early results on oblique projections, which foreshadowed the derivation of (1.1) can be found in [1]; see also [3], [10], [13], [20], [37], [40], [44], for some additional properties of oblique projections not discussed here.…”

Jordan's principal angles in complex vector spaces

“…Property (13) and the following consequence also appear in Reference [7] for real vector spaces. Afriat called the pairs (u i , v i ) reciprocal.…”

“…Theorem 11 (Afriat [7,23], Gutmann and Shepp [24]) In any pair of subspaces M 1 and M 2 there exist orthonormal bases…”

“…The importance of the matter and Hotelling's paper [6] initiated many further investigations such as References [7][8][9].…”

The Angle Between Complementary Subspaces

“…Perhaps this is because most instructors feel that such extensions are difficult to understand, or that further effort in this direction is not worthwhile. Indeed, this makes sense for angles between general subspaces because one would have to introduce concepts like gap or distance between subspaces [Kato (1966), Stewart and Sun (1990)], principal (or canonical) angles [Afriat (1957), Björck and Golub (1973), Wedin (1982), Stewart and Sun (1990)], the CS decomposition [Stewart (1977), Davis and Kahan (1970), Stewart (1973), Golub and Van Loan (1989), Stewart and Sun (1990)], and so on. These topics are better off in a more advanced course.…”