Batch processing and data streaming Fourier-based convolutional neural network accelerator

Abstract

Decision-making through artificial neural networks with minimal latency is critical for numerous applications such as navigation, tracking, and real-time machine action systems. This requires machine learning hardware to process multidimensional data at high throughput. Unfortunately, handling convolution operations, the primary computational tool for data classification tasks, obeys challenging runtime complexity scaling laws. However, homomorphically implementing the convolution theorem in a Fourier optics display light processor can achieve a non-iterative (1) runtime complexity for data inputs beyond 1,000 × 1,000 large matrices. Following this approach, here we demonstrate data streaming multi-kernel image batching using a Fourier Convolutional Neural Network (FCNN) accelerator. We show image batch processing of large-scale matrices as 2 million dot product multiplications performed by a digital light processing module in the Fourier domain. Furthermore, we further parallelize this optical FCNN system by exploiting multiple spatially parallel diffraction orders, achieving a 98x throughput improvement over state-of-the-art FCNN accelerators. A comprehensive discussion of the practical challenges associated with working at the edge of system capabilities highlights the problem of crosstalk and resolution scaling laws in the Fourier domain. Accelerating convolution by exploiting massive parallelism in display technology brings non-Van Neumann-based machine learning acceleration.

Decentralized power grid control scheme utilizing photonic sensing and computing

“…This technique typically employs CNNs and therefore can be replaced with a fully connected photonic CNN. 23…”

Free-space optical multiplexed orbital angular momentum beam identification system using Fourier optical convolutional layer based on 4f system

“…After two decades development, kernels generated by state-in-the-art DMD outperforms the older hologram mask with a higher enough throughput to simulate completely neural planes, makes optical convolutional efficiency promising. [87][88][89][90][91][92][93][94][95][96][97][98][99][100][101] In this study, we experimentally generate multiplexed OAM beams and introduce optical filtering method which relys on spatial Fourier transform of images in the frequency domain as optical convolutional neural network to train and identify multiplexed OAM beams under simulated atmospheric turbulence conditions, we show the system currently capable of classifying 12 classes at test accuracy of 95% (under weak turbulence) and 87% (under strong turbulence).…”

High-performance optoelectronics for integrated photonic neural networks

“…Artificial neurons based on nanophotonic technologies can potentially provide the platform that can fulfill the challenging future technological needs. [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] Integrated photonic technology provides a solution to the limitations of current digital electronic counterparts like efficient fundamental computational operations such as weighted sum or addition, vector matrix multiplications, or convolutions technologically enabled by attojoule efficient electro-optic (EO) modulators, phase shifters, and combiners. Furthermore, high parallelism and bandwidth is provided by exploiting wavelength-, polarization-and/or mode-division multiplexing.…”