Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks

Abstract

We introduce an electro-optic hardware platform for nonlinear activation functions in optical neural networks. The optical-to-optical nonlinearity operates by converting a small portion of the input optical signal into an analog electric signal, which is used to intensity-modulate the original optical signal with no reduction in processing speed. Our scheme allows for complete nonlinear on-off contrast in transmission at relatively low optical power thresholds and eliminates the requirement of having additional optical sources between each layer of the network. Moreover, the activation function is reconfigurable via electrical bias, allowing it to be programmed or trained to synthesize a variety of nonlinear responses. Using numerical simulations, we demonstrate that this activation function significantly improves the expressiveness of optical neural networks, allowing them to perform well on two benchmark machine learning tasks: learning a multi-input exclusive-OR (XOR) logic function and classification of images of handwritten numbers from the MNIST dataset. The addition of the nonlinear activation function improves test accuracy on the MNIST task from 85% to 94%.

Universal programmable photonic architecture for quantum information processing

“…In the previous section we discussed preparation of arbitrary quantum states or operators by obtaining appropriate phase shifter values to implement an exact decomposition of the desired operation using only singlequbit and nearest-neighbor cσ z gates. In this section, we demonstrate a method, building on our previous work for classical MZI networks [13,14] and on work for continuous-variable quantum neural networks [51], of automatically discovering high-fidelity approximate decompositions of a target operator using a gradient-based optimization approach. As shown in Section IV A 4, these "learned" implementations of quantum operators are often far more compact than an explicit decomposition, allowing for lattices with a fraction of the physical depth.…”

“…Such devices have a wide range of applications in classical information processing [4,[6][7][8][9][10], and integrated universal photonic circuits provides an especially promising hardware platform for high-throughput, energy-efficient machine learning. [11][12][13][14] These devices also have promising applications in quantum information processing: recent demonstrations of boson sampling [15], quantum transport dynamics [16], photonic quantum walks [17], counterfactual communication [18], and probabilistic two-photon gates [19] have all been performed on this type of programmable photonic hardware. Photonic systems offer a range of unique advantages over other substrates for quantum information processing: optical quantum states have long coherence times and can be maintained at room temperature, since they interact very weakly with their environment; photonic qubits are optimal information carriers for distant nodes within quantum networks; and MZIs provide simple, high-fidelity implementations of single-qubit operations which can be integrated into a photonic chip.…”

Photonic convolutional neural networks using integrated diffractive optics

“…Recent advances in photonic integration have enabled demonstration of low-energy and high speed optical-electricaloptical or O-E-O devices, which act as pseudo-optical nonlinear devices [48]. Such O-E-O devices are reconfigurable to show a variety of output responses, including an approximation of the widely used ReLU function [49], [50]. Figure 3 shows a comparison of a typical CNN architecture schematic and an equivalent photonic CNN implementation.…”

Efficient training and design of photonic neural network through neuroevolution

“…However, the BP and SGD training strategies were difficult to implement in the integrated optical chips, thus the determination of the weights in the ONNs were generally mapped from the pre-trained results on a digital computer [25]. Obviously, this train method was inefficient because of the restricted accuracy of the model representation and the loss of the advantages in speed and energy [31].In order to self-learn the weights in the ONNs, in situ computation of the gradient for weights based on brute force had been reported [25]. Similarly, a self-learning photonic signal processor trained by a modified SGD was proven experimentally to implement tunable filter, optical switching and descrambler [32].…”

“…Here, the determinations of the gradients in the ONNs were converted to an inverse design and sensitivity analysis process for photonic circuits [33]. In addition, some training strategies and tricks used in deep learning, such as adaptive moment estimation and initialization scheme were also applied in training of the ONNs [29,31]. The training methods proposed by T.W.…”