On the Early History of the Singular Value Decomposition

Continuous analogues of matrix factorizations

“…This fundamental property of the SVD originates with Schmidt [3,15,16] and was generalized to rectangular matrices by Eckart & Young [17]. The SVD also enjoys an optimality property in the 2-norm and other unitarily invariant norms, but our discussion of continuous analogues will be confined to the Frobenius norm.…”

“…We are grateful for suggestions by Anthony Austin, Mario Bebendorf, Hrothgar, Sabine Le Borne, Colin Macdonald, Cleve Moler and Gil Strang. In addition, we would like to acknowledge the outstanding contributions of Pete Stewart over the years in explicating the historical and mathematical roots of the SVD and other matrix ideas, including his annotated translations of the key papers of Fredholm et al [5,15]. The second author thanks Martin Gander of the University of Geneva for hosting a sabbatical visit during which much of this article was written.…”

Global and local distance-based generalized linear models

“…Since for n cases, depending on the chosen metric, r can be as high as n − 1, giving an overparametrized model with unstable predictions, a sensible procedure is to replace the pseudo-inverse F + w with a lower-rank approximation. This can be easily implemented by the Singular Value Decomposition which, by the Schmidt-Eckart-Young Theorem (see, e.g., Stewart 1993), gives the best 2 approximation of any given rank k, 1 ≤ k ≤ r. The rank k used to define the pseudo-inverse F + w is called effective rank. Several criteria can be used to select a suitable value for effective rank k: ordinary or generalized crossvalidation (OCV or GCV), as well as Akaike or Bayesian information criterion (AIC or BIC), defined as in the ordinary linear model (LM).…”

Balanced Realizations, Model Order Reduction, and the Hankel Operator

“…The analysis is important since it extracts the gain structure of the operator, that is, it characterizes the largest input-output ratio and the corresponding input [51]. Since linear singular values are defined as eigenvalues of the composition of the given operator and its adjoint, it is natural to introduce a nonlinear version of adjoint operators to obtain a nonlinear counterpart of a singular value.…”