Diffraction-driven parallel convolution processing with integrated photonics

Abstract

Traditional electronic processors face challenges such as bandwidth limitations and high power consumption when handling extensive linear operations for deep learning tasks. As an alternative, optical computing has garnered attention for its potential in parallel and energy-efficient computations. However, the exploration of high-density optical computing architectures on integrated photonic platforms has been limited, primarily due to constraints in neuron numbers and control engineering complexities. In response to these challenges, we report a diffraction-driven multi-kernel optical convolution unit (MOCU) to enable on-chip parallel convolution processing. Utilizing cascaded silica 1D metalines as pre-trained large-scale weights and employing spatial multiplexing at the output facet, MOCU enables simultaneous computing of diverse convolutions passively within one single unit. With MOCU, we build optical convolutional neural networks (OCNNs), enabling efficient processing of machine visions with concise architecture. To address fabrication errors inherent in MOCU-embedded OCNN, a lightweight electronic neural network runs concurrently with the OCNN to calibrate systematic deviations via low-rank adaptation (LoRA) algorithm with negligible overhead. The fabricated MOCU chip achieves the highest independent 8-kernel convolutions in parallel, with each kernel size of 3 × 3 and occupying a footprint of merely 0.06mm^2. The proposed architecture synergistically leverages the benefits of both photonic and electronic technologies, offering a potential design methodology for next-generation deep learning hardware aimed at achieving high compute density and prominent energy efficiency.