Adjoint Differentiation for generic matrix functions

Abstract

We derive a formula for the adjoint A of a square-matrix operation of the form C = f (A), where f is holomorphic in the neighborhood of each eigenvalue. We then apply the formula to derive closed-form expressions in particular cases of interest such as the case when we have a spectral decomposition A = U DU −1 , the spectrum cut-off C = A + and the Nearest Correlation Matrix routine. Finally, we explain how to simplify the computation of adjoints for regularized linear regression coefficients.