Neuromorphic Silicon Photonics and Hardware-Aware Deep Learning for High-Speed Inference

Moralis-Pegios, Miltiadis; Mourgias-Alexandris, George; Tsakyridis, Apostolos; Giamougiannis, George; Totović, Angelina; Dabos, George; Passalis, Nikolaos; Kirtas, Manos; Rutirawut, Teerapat; Gardes, Frederic Y.; Tefas, Anastasios; Pleros, Nikos

doi:10.1109/jlt.2022.3171831

Cited by 41 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…20,21 Remarkable progress has been witnessed during the last five years in the field of neuromorphic photonics across all necessary constituent technology blocks, including MVM photonic architectures, 17,[22][23][24][25][26][27][28] individual photonic computational elements, [29][30][31][32] nonlinear activations, [33][34][35][36] and photonic hardware-aware training models. 37,38 All these demonstrations have highlighted the potential for energyefficient and high-speed DNNs by utilizing low-speed weight encoding technologies and a rather small amount of neurons, validating their credentials to support inference within small scale neural network (NN) topologies that can fit in a practical silicon photonic (SiPho) chip.…”

Section: Introductionmentioning

confidence: 91%

“…Yet, as transistor scaling is stagnating, 10 a high number of alternative emerging technologies have been investigated toward boosting energy efficiency and performance scaling, e.g., optoelectronic memristors, 11 – 15 nanophotonics, 16 , 17 and spintronics, 18 , 19 with brain-inspired photonic accelerators forming one of the key candidate platforms for future AI computing engines due to their inherent credentials to support time-of-flight latencies and terahertz bandwidths 20 , 21 . Remarkable progress has been witnessed during the last five years in the field of neuromorphic photonics across all necessary constituent technology blocks, including MVM photonic architectures, 17 , 22 – 28 individual photonic computational elements, 29 – 32 nonlinear activations, 33 – 36 and photonic hardware-aware training models 37 , 38 . All these demonstrations have highlighted the potential for energy-efficient and high-speed DNNs by utilizing low-speed weight encoding technologies and a rather small amount of neurons, validating their credentials to support inference within small scale neural network (NN) topologies that can fit in a practical silicon photonic (SiPho) chip.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications

et al. 2023

Self Cite

View full text Add to dashboard Cite

The explosive volume growth of deep-learning (DL) applications has triggered an era in computing, with neuromorphic photonic platforms promising to merge ultra-high speed and energy efficiency credentials with the brain-inspired computing primitives. The transfer of deep neural networks (DNNs) onto silicon photonic (SiPho) architectures requires, however, an analog computing engine that can perform tiled matrix multiplication (TMM) at line rate to support DL applications with a large number of trainable parameters, similar to the approach followed by state-of-the-art electronic graphics processing units. Herein, we demonstrate an analog SiPho computing engine that relies on a coherent architecture and can perform optical TMM at the record-high speed of 50 GHz. Its potential to support DL applications, where the number of trainable parameters exceeds the available hardware dimensions, is highlighted through a photonic DNN that can reliably detect distributed denial-of-service attacks within a data center with a Cohen's kappa score-based accuracy of 0.636.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Introductionmentioning

confidence: 99%

Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is worth noting that many of the limitations encountered in the PNN can be eventually alleviated by enforcing a hardware-aware DL training framework where the training algorithm incorporates the physical layer limitations of the underlying photonic hardware a priori in the training process [8,9]. Accounting for quantization (limited bit resolution) [10,11], limited ER or bandwidth has already been demonstrated as a viable solution for performance upgrade [8,9].…”

Section: Network Performance Analysismentioning

confidence: 99%

“…Having most real-world information analog in nature supports the trend of transitioning from digital domain to the analog one, especially in the light of recent advances in low precision [6] and noise-resilient training algorithms [7][8][9][10][11]. The superiority of the electronic crossbars has already been recognized by Syntiant [12], MemryX [13] and Mythic [14], with photonic platforms striving to produce an equivalent circuit for analog neuromorphic photonic setups, as shown by the works of Lightmatter [15,16] and Lightelligence [17,18] in commercial-grade large-scale coherent photonic integrated circuits (PICs).…”

Section: Introductionmentioning

confidence: 99%

WDM equipped universal linear optics for programmable neuromorphic photonic processors

Totović¹,

Pappas²,

Kirtas³

et al. 2022

Neuromorph. Comput. Eng.

Self Cite

View full text Add to dashboard Cite

Non-von-Neumann computing architectures and Deep Learning training models have sparked a new computational era where neurons are forming the main architectural backbone and vector, matrix and tensor multiplications comprise the basic mathematical toolbox. This paradigm shift has triggered a new race among hardware technology candidates; within this frame, the field of neuromorphic photonics promises to convolve the targeted algebraic portfolio along a computational circuitry with unique speed, parallelization, and energy efficiency advantages. Fueled by the inherent energy efficient analog matrix multiply operations of optics, the staggering advances of photonic integration and the enhanced multiplexing degrees offered by light, neuromorphic photonics has stamped the resurgence of optical computing brining a unique perspective in low-energy and ultra-fast linear algebra functions. However, the field of neuromorphic photonics has relied so far on two basic architectural schemes, i.e., coherent linear optical circuits and incoherent WDM approaches, where wavelengths have still not been exploited as a new mathematical dimension. In this paper, we present a radically new approach for promoting the synergy of WDM with universal linear optics and demonstrate a new, high-fidelity crossbar-based neuromorphic photonic platform, able to support matmul with multidimensional operands. Going a step further, we introduce the concept of programmable input and weight banks, supporting in situ reconfigurability, forming in this way the first WDM-equipped universal linear optical operator and demonstrating different operational modes like matrix-by-matrix and vector-by-tensor multiplication. The benefits of our platform are highlighted in a Fully Convolutional Neural Network layout that is responsible for parity identification in the MNIST handwritten digit dataset, with physical layer simulations revealing an accuracy of ~94%, degraded by only 2% compared to respective results obtained when executed entirely by software. Finally, our in-depth analysis provides the guidelines for neuromorphic photonic processor performance improvement, revealing along the way that 4-bit quantization is sufficient for inputs, whereas the weights can be implemented with as low as 2-bits of precision, offering substantial benefits in terms of driving circuitry complexity and energy savings.

show abstract

“…Typical PNNs include fiber optic networks, [8][9][10] free space optic networks, [11,12] and integrated photonic circuits. [13][14][15] Each solution has its own characteristics, but they all share the same goal of exploiting the potential benefits of PNNs' large bandwidth and high energy efficiency [13,[16][17][18][19][20][21] to break bread with electronic digital processors represented by graphics processing units in the third artificial intelligence boom. However, increasing problems have been exposed with increasing related studies on PNN.…”

Section: Introductionmentioning

confidence: 99%

Dual Optical Frequency Comb Neuron: Co‐Developing Hardware and Algorithm

Zhang

Tao

Yan

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Previous studies on photonic neural network have demonstrated that algorithm can inspire hardware design. This study seeks to demonstrate that hardware can also inspire algorithm design. To further exploit the advantages of photonic analog computing, the authors develop hardware and algorithm simultaneously for photonic convolutional neural networks. Specifically, this work developed an architecture called dual optical frequency comb neuron (DOFCN) enabled by an integrated microcomb to perform cosinusoidal nonlinear activation and vector convolution without temporal or spatial dispersion and large‐scale modulator arrays. Furthermore, DOFCN‐based composite vector convolutional neural networks (CVCNNs), an optical‐electric hybrid model, are proposed to perform classification and regression tests in signal modulation format identification and optical structure inverse design tasks, respectively. The ablation experiments show that under 4‐bit precision limit, the element‐wise activation CVCNN has 14% higher classification accuracy, 76% lower regression residuals, and 100% higher training efficiency than that of the 32‐bit standard convolutional neural network (CNN). DOFCN exhibits impressive spectral information processing ability to facilitate signal‐processing tasks related to optics and electromagnetics.

show abstract

Neuromorphic Silicon Photonics and Hardware-Aware Deep Learning for High-Speed Inference

Cited by 41 publications

References 41 publications

Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications

Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications

WDM equipped universal linear optics for programmable neuromorphic photonic processors

Dual Optical Frequency Comb Neuron: Co‐Developing Hardware and Algorithm

Contact Info

Product

Resources

About