Ultralow Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision

Abstract

In-sensor and near-sensor computing architectures enable multiply-accumulate operations to be carried out directly at the point of sensing. In-sensor architectures offer dramatic power and speed improvements over traditional von Neumann architectures by eliminating multiple analog-todigital conversions, data storage, and data movement operations. Current in-sensor processing approaches rely on tunable sensors or additional weighting elements to perform linear functions such as multiply-accumulate operations as the sensor acquires data. We implement in-sensor computing with an oscillatory retinal neuron device that converts incident optical signals into voltage oscillations. We introduce a computing scheme based on the frequency shift of coupled oscillators that enables parallel, frequency multiplexed, non-linear operations on the inputs. Simulation elucidates how this computing occurs. We experimentally implement a 3×3 focal plane array of coupled neurons and show that functions approximating edge detection, thresholding, and segmentation occur in parallel. An example of inference on handwritten digits from the MNIST database is also experimentally demonstrated with a 3×3 array of coupled neurons feeding into a