High-density Integrated Photonic Tensor Processing Unit with a Matrix Multiply Compiler

Abstract

Machine learning and artificial intelligence (AI) is becoming a ubiquitous technology of the looming industry-4.0 era. However, progress of adopting intelligent automation of systems is limited by hardware overhead such as throughput, power consumption, and latency. At a conceptual level, electronics is at the end of its scaling law and alternative accelerators are sought after. Optical co-processors offer a high-degree of algorithmic homomorphism to implement general matrix-matrix multiplication operations via on-the-fly multiplication performed by electro-optic components, and accumulation operations performed by photodetectors. However, recent emerging photonic AI engines are cumbersome to program, follow overhead-heavy scaling laws, or rely on discretely-packaged photonic components, all reducing performance. Here we introduce a photonic tensor core processor featuring a complete chip-integrated multiply-accumulate engine on the photonic circuits, signal parallelism via wavelength division multiplexing, and tunable 5-bit electronic programmable weights, all integrated into a small-formfactor silicon photonics platform. This approach shows a high throughput-efficiency AI engine (0.21 TOPS/W). Using this stand-alone hybrid photonic-electronic machine learning accelerator, we demonstrate versatile applications including low-error image filtering and edge detection, augmented reality, and high-accuracy machine learning image classification tasks. This hybrid electronic-photonic tensor processor offers a versatile low-overhead and compact formfactor solution for extreme edge AI applications, but also can be utilized for in-cloud machine learning training tasks.

Photonic tensor core for machine learning: a review

“…31 An alternative scheme has been proposed by Miscuglio et al, 32 where the weights are implemented between a series of demux and mux add-drop microring resonators. This architecture has been proven by Ma et al 27 with tunable low-speed MZI acting as weights, and by using PCM components. 33 This double approach permits to adap of t the circuit to the applications of the NN, for example, tunable high-speed weights can be used in Cloud applications, where training requires a high rate of updates, while the PCM solutions, exploiting the non-volatility, can be adapt to edge computing, where energy efficiency is a major concern.…”

High-performance optoelectronics for integrated photonic neural networks

“…Artificial neurons based on nanophotonic technologies can potentially provide the platform that can fulfill the challenging future technological needs. [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] Integrated photonic technology provides a solution to the limitations of current digital electronic counterparts like efficient fundamental computational operations such as weighted sum or addition, vector matrix multiplications, or convolutions technologically enabled by attojoule efficient electro-optic (EO) modulators, phase shifters, and combiners. Furthermore, high parallelism and bandwidth is provided by exploiting wavelength-, polarization-and/or mode-division multiplexing.…”

Michelson interferometric methods for full optical complex convolution

“…[41][42][43][44][45][46][47][48][49] At the same time, in optics, the Optical Fourier Transform (FT) and dot-product multiplication performed passively by a single lens or metalens have reduced computational complexity compared to logarithmic scaling in electronic processing units, [50][51][52][53][54][55][56] both free-space [57][58][59][60][61][62][63][64][65] and integrated photonic integrated circuits versions. [66][67][68][69][70][71][72][73][74][75][76][77][78][79][80] The only drawback of Spatial Light Modulators (SLMs) based optical computing systems would be the refresh rate of the device itself, but with recent research progress on novel phase-changing material and other 2D material as well as their corresponding applications, [81][82][83][84][85][86]…”