Massively parallel amplitude-only Fourier neural network

Abstract

Machine intelligence has become a driving factor in modern society. However, its demand outpaces the underlying electronic technology due to limitations given by fundamental physics, such as capacitive charging of wires, but also by system architecture of storing and handling data, both driving recent trends toward processor heterogeneity. Task-specific accelerators based on free-space optics bear fundamental homomorphism for massively parallel and real-time information processing given the wave nature of light. However, initial results are frustrated by data handling challenges and slow optical programmability. Here we introduce a novel amplitude-only Fourier-optical processor paradigm capable of processing large-scale ∼ ( 1000 × 1000 ) matrices in a single time step and 100 µs-short latency. Conceptually, the information flow direction is orthogonal to the two-dimensional programmable network, which leverages 10 6 parallel channels of display technology, and enables a prototype demonstration performing convolutions as pixelwise multiplications in the Fourier domain reaching peta operations per second throughputs. The required real-to-Fourier domain transformations are performed passively by optical lenses at zero-static power. We exemplary realize a convolutional neural network (CNN) performing classification tasks on 2 megapixel large matrices at 10 kHz rates, which latency-outperforms current graphic processing unit and phase-based display technology by 1 and 2 orders of magnitude, respectively. Training this optical convolutional layer on image classification tasks and utilizing it in a hybrid optical-electronic CNN, shows classification accuracy of 98% (Modified National Institute of Standards and Technology) and 54% (CIFAR-10). Interestingly, the amplitude-only CNN is inherently robust against coherence noise in contrast to phase-based paradigms and features a delay over 2 orders of magnitude lower than liquid-crystal-based systems. Such an amplitude-only massively parallel optical compute paradigm shows that the lack of phase information can be accounted for via training, thus opening opportunities for high-throughput accelerator technology for machine intelligence with applications in network-edge processing, in data centers, or in pre-processing information or filtering toward near-real-time decision making.

Quantifying Information via Shannon Entropy in Spatially Structured Optical Beams

“…The constraints of probability theory on the definition of information and of quantum mechanics on conjugate observables are satisfied working around the properties of the WDF-a pseudo-probability distribution. We foresee the relevance of this formalism in the context of recent developments for (i) free-space information-processing optics [34]; (ii) integrated photonics-based information processing [35] such as neural network-based accelerators [36] and photonic tensor cores [37]; (iii) adaptive sensing [38]; and (iv) analog optical and photonic processors [39][40][41]. As the data compression coefficient is naturally bounded by Shannon information, carried by the beam [42], this work indirectly points towards higher information capacity in beams with a nontrivial structure, like HG, LG, and Bessel-Gauss modes [43,44].…”

“…In perspective, as the demand on high-speed data transfer and streaming grows exponentially, ADSL and fiber-tohome technologies are less and less likely to satisfy even an average consumer's data hunger, not to mention business and government agency calls. These, together with the recent advents in optical processing [34], micro-and nanofabrication [45], and OAM communications [46] put forward the mid-20th century's excitement around free-space communications in a new light. The new-generation free-space links will require coherent detection techniques to realise their potential to the fullest.…”

Photonic extreme learning machine by free-space optical propagation

“…Our freespace PELM hence surpasses diffractive neural networks [8] and competes with cutting-edge photonic hardware in terms of accuracy. In particular, convolutional opto-electronic setups also achieve a coarse accuracy of 92% [19]. Ultrafast photonic processors reach 95% precision with the help of electronic layers [15], whereas similar photonic accelerators operate with 88% accuracy [16].…”

“…We benchmark the realized device on problems of different classes, achieving performance comparable with digital ELMs. These include a classification accuracy exceeding 92% on the well-known MNIST database, which overcomes fabricated diffractive neural networks [8]; further, it is comparable with convolutional artificial networks that employ photonic accelerators [15,16,19]. Given the massive parallelism provided by spatial optics and the ease of training, our approach is ideal for big data, i.e., extensive data sets with large dimension samples.…”

“…Pioneering attempts using photosensitive masks [4] and volume holograms [5,6] have been recently developed into coherent optical neural networks that promise fast and energy-efficient optical computing [7][8][9][10][11][12][13][14][15][16]. These schemes exploit optical units such as nanophotonic circuits [7], on-chip frequency combs [14][15][16], and spatial light modulators to perform matrix multiplications in parallel [17] or to carry out convolutional layers [18][19][20]. Training consists in adjusting the optical response of each physical node [21], also by adopting external optical signals [22], which is demanding [23].…”