We use the exterior product of double forms to reformulate celebrated classical results of linear algebra about matrices and bilinear forms namely Cayley-Hamilton theorem, Laplace expansion of the determinant, Newton identities and Jacobi's formula for the determinant. This new formalism is then used to naturally generalize the previous results higher multilinear forms namely to double forms. In particular, we show that the Cayley-Hamilton theorem once applied to the second fundamental form of a hypersurface is equivalent to a linearized version of the Gauss-Bonnet theorem, and once its generalization is applied to the Riemann curvature tensor (seen as a (2, 2) double form) is an infinitisimal version of the general Gauss-Bonnet-Chern theorem. In addition to that, the general Cayley-Hamilton theorems generate several universal curvature identities. The generalization of the classical Laplace expansion of the determinant to double forms is shown to lead to new general Avez type formulas for all Gauss-Bonnet curvatures.