Efficient training and design of photonic neural network through neuroevolution

Abstract

Recently, optical neural networks (ONNs) integrated in photonic chips has received extensive attention because they are expected to implement the same pattern recognition tasks in the electronic platforms with high efficiency and low power consumption. However, the current lack of various learning algorithms to train the ONNs obstructs their further development. In this article, we propose a novel learning strategy based on neuroevolution to design and train the ONNs. Two typical neuroevolution algorithms are used to determine the hyper-parameters of the ONNs and to optimize the weights (phase shifters) in the connections. In order to demonstrate the effectiveness of the training algorithms, the trained ONNs are applied in the classification tasks for iris plants dataset, wine recognition dataset and modulation formats recognition. The calculated results exhibit that the training algorithms based on neuroevolution are competitive with other traditional learning algorithms on both accuracy and stability. Compared with previous works, we introduce an efficient training method for the ONNs and demonstrate their broad application prospects in pattern recognition, reinforcement learning and so on. IntroductionArtificial neural networks (ANNs), deep learning [1] in particular, has attracted a great deal of research attentions for an impressively large number of applications, such as image processing [2], natural language processing [3], acoustical signal processing [4], time series processing [5], self-driving [6], games [7], robot [8] and so on. It should be noted that the training of the ANNs with deep hidden layers, especially for convolutional neural networks (CNNs) and recurrent neural networks (RNNs), for example AlexNet [9], VGGNet [10], GoogLeNet [11], ResNet [12] and long short-term memory [13], typically demands significant computational time and resources [1]. Thus, various electronic special-purpose platforms based on graphical processing units (GPUs) [14], field-programmable gate arrays (FPGAs) [15] and applicationspecific integrated circuits (ASICs) [16] were invented to accelerate the training and inference process of deep learning. On the other hand, in order to obtain general artificial intelligence, some brain-inspired chips including IBM TrueNorth [17], Intel Loihi [18], and SpiNNaker [19]were designed by imitating the structure of a brain. However, even both energy efficiency and speed were improved, the performances of the brain-inspired chips were difficult to compete with the state of the art of deep learning [20]. In the recent years, optical computing had been demonstrated as an effective alternative to traditionally electronic computing architectures and expected to alleviate the bandwidth bottlenecks and power consumption in electronics [21]. For example, new photonic approaches for spiking neuron and scalable network architecture based upon excitable lasers, broadcast-and-weight protocol and reservoir computing had been illustrated [22][23][24]. Despite ultrafast spiking response were achiev...

FLOPS: EFficient On-Chip Learning for OPtical Neural Networks Through Stochastic Zeroth-Order Optimization

“…Adjoint variable method (AVM) (Hughes et al 2018) is proposed to model the circuit state as a partial-differential-equation-controlled linear system, and directly measures the exact gradient via in situ light intensity measurement. Evolutionary algorithms, e.g., particle swarm optimization and genetic algorithm, are demonstrated to train MZIs on chip (Zhang et al 2019). A stochastic zeroth-order gradient descent based method FLOPS (Gu et al 2020a) has been proposed to improve the training efficiency by 3-5× compared with previous methods.…”

Deep Learning Enabled Nanophotonics

“…Recently, another approach to realise optical neural networks was based on Mach-Zehnder interferometers (MZIs) to calculate matrix products [25,26], see Figure 6b. By carefully manipulating a specific phase shift between a coherent pair of incoming light pulses allow to multiply a two-element vector, encoded in the amplitude of the pulses, by a two-by-two matrix [27,28]. An array of the interferometers can then perform arbitrary matrix operations, which is widely used, for example, in the boson sampling approach.…”

Efficient Optical Spatial First-Order Differentiator Based on Graphene-Based Metalines and Evolutionary Algorithms

“…In addition, the high tunability of graphene metalines can be combined with inverse design technology to design a more efficient photonics device [28]- [30]. In general, inverse design is converted to the optimization problem which can be solved by gradient based methods (such as adjoint variable method (AVM) [31]- [32]), gradient free methods (such as genetic algorithm (GA) [33]- [37]) and model based methods (such as machine learning [38]- [40]). As a representative algorithm of gradient based methods, AVM not only designs linear devices but also optimizes for the nonlinear devices in frequency domain, but it requires physical background to derive the gradient of objective function [31]- [32].…”

“…However, in order to train the model whose inputs are physical parameters and outputs are electromagnetic responses, it requires a significant amount of time to generate the training instances and test instances [38]. In comparison to the gradient based methods and model based methods, gradient free methods, which depend on search strategy and evolutionary strategy, are simple, effective and parallelizable [33]- [37]. As a result, although GA easily falls into local optimum and demands tremendous computational time, it has been applied in the inverse design for many photonics devices, such as polarization beam splitters [33], polarization rotators [35], diodes [34], waveguide crossings [36], optical neural networks [37] and so on.…”

“…In comparison to the gradient based methods and model based methods, gradient free methods, which depend on search strategy and evolutionary strategy, are simple, effective and parallelizable [33]- [37]. As a result, although GA easily falls into local optimum and demands tremendous computational time, it has been applied in the inverse design for many photonics devices, such as polarization beam splitters [33], polarization rotators [35], diodes [34], waveguide crossings [36], optical neural networks [37] and so on. It should be noted that previous researches pay little attention to the inverse design and performance optimization of OSD and graphene-based photonics devices, especially for the optimization of the physical parameters for graphene metasurfaces, such as chemical potential, relaxation time and so on.…”