Neural Schrödinger Equation: Physical Law as Deep Neural Network

Abstract

We show a new family of neural networks based on the Schrödinger equation (SE-NET). In this analogy, the trainable weights of the neural networks correspond to the physical quantities of the Schrödinger equation. These physical quantities can be trained using the complex-valued adjoint method. Since the propagation of the SE-NET can be described by the evolution of physical systems, its outputs can be computed by using a physical solver. The trained network is transferable to actual optical systems. As a demonstration, we implemented the SE-NET with the Crank-Nicolson finite difference method on Pytorch. From the results of numerical simulations, we found that the performance of the SE-NET becomes better when the SE-NET becomes wider and deeper. However, the training of the SE-NET was unstable due to gradient explosions when SE-NET becomes deeper. Therefore, we also introduced phase-only training, which only updates the phase of the potential field (refractive index) in the Schrödinger equation. This enables stable training even for the deep SE-NET model because the unitarity of the system is kept under the training. In addition, the SE-NET enables a joint optimization of physical structures and digital neural networks. As a demonstration, we performed a numerical demonstration of end-to-end machine learning (ML) with an optical frontend toward a compact spectrometer. Our results extend the application field of ML to hybrid physical-digital optimizations.

Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware

“…The training using Eq. ( 1 ) is typically executed on a regular external computer by constructing a physical simulation model 14 , 16 , 30 , 39 , 44 , which incurs large computational cost. Thus, this strategy is not suitable for in-situ training.…”

“…First, the physical implementations of the BP operation are still complex and unscalable 40 – 43 . Thus, the calculation for BP for a PNN is typically executed on an external regular computer with a simulation model of a physical system 14 , 16 , 30 , 39 , 44 . This strategy results in a loss of any advantage in speed or energy associated with using the physical circuit in the training process.…”

“…This framework is called a physical reservoir computer (RC) [21][22][23] or an extreme learning machine (ELM) [24][25][26] , whose ease of implementation has greatly expanded the choice of implementable materials and its application range. Such physically implemented neural networks (PNNs) enable the outsourcing of the computational load for specific tasks to a physical system such as a memory 27 , optical link 28,29 , sensor component 30,31 , or robotic body 32 . The experimental demonstrations of these unconventional computations have revealed performance competitive with that of conventional electronic computing [33][34][35] .Constructing deeper physical networks is one promising direction for further performance improvement because they can extend network expression ability exponentially 36,37 , as opposed to the polynomial relationship in wide (large-node-count) networks.…”

“…The experimental demonstrations of these unconventional computations have revealed performance competitive with that of conventional electronic computing [33][34][35] .Constructing deeper physical networks is one promising direction for further performance improvement because they can extend network expression ability exponentially 36,37 , as opposed to the polynomial relationship in wide (large-node-count) networks. This has motivated proposals of deep PNNs using various physical platforms 14,16,30,[38][39][40][41][42][43] .Their training has basically relied on a method called backpropagation (BP), which has seen great success in the software-based ANN.…”

Observing how deep neural networks understand physics through the energy spectrum of one-dimensional quantum mechanics

“…Partial differential equations, including the Schrödinger equation, have been solved using NNs [8], potentials have been estimated inversely from wave functions [9,10], and soliton solutions to the nonlinear Schrödinger equation have been investigated using NNs [11]. A new family of NNs inspired by the Schrödinger equation has also been proposed [12].…”