Unitary learning in conditional modes for deep optics neural networks

Abstract

Complex-valued backprogation (CVBP) is an essential algorithm parsing a large amount of complex data for statistical inferences to deep optics neural networks. However, the operating optical transformations are almost unitary, which require the unitary backpropagation algorithms to meet the hetero-equivalence from their mathematical to optical mechanisms. This paper mainly compares two compact formulations for a common CVBP and its unitary-variants under holomorphic and compatible conditions, for which composite mechanisms are invented. For unitary weights-update, essentially, the Riemannian gradient of a CVBP is expanded from Euclidean gradient by executing an exponential mapping onto its unitary manifold. The convergent speed and accuracy, as well as the evolutions of their energies and averaging-phases at each layer are progressively investigated and compared, which reflect the dynamic behaviors on the network convergences. We find that not all the nonlinear activations under the respective conditions are always ensured to convergences, and the averaging-phases still remain several jumps although the energies at current layer situate at the convergent districts.