PISA: A Binary-Weight Processing-In-Sensor Accelerator for Edge Image Processing

Angizi, Shaahin; Tabrizchi, Sepehr; Roohi, Arman

doi:10.48550/arxiv.2202.09035

Cited by 5 publications

(11 citation statements)

References 40 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Systematic integration of computing and sensor arrays has been widely studied to eliminate off-chip data transmission and reduce ADC bandwidth by combining CMOS image sensors and processors in one chip, known as PNS [8][9][10][11][12], or even integrating pixels and computation unit so-called PIS [13][14][15][16]. In [9], photocurrents are transformed into pulse-width modulation signals, and a dedicated analog processor is designed to execute feature extraction reducing ADC power consumption.…”

Section: Near/in-sensor Processing Backgroundmentioning

confidence: 99%

Ocelli: Efficient Processing-in-Pixel Array Enabling Edge Inference of Ternary Neural Networks

Tabrizchi

Angizi

Roohi

2022

JLPEA

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs), due to their recent successes, have gained lots of attention in various vision-based applications. They have proven to produce incredible results, especially on big data, that require high processing demands. However, CNN processing demands have limited their usage in embedded edge devices with constrained energy budgets and hardware. This paper proposes an efficient new architecture, namely Ocelli includes a ternary compute pixel (TCP) consisting of a CMOS-based pixel and a compute add-on. The proposed Ocelli architecture offers several features; (I) Because of the compute add-on, TCPs can produce ternary values (i.e., −1, 0, +1) regarding the light intensity as pixels’ inputs; (II) Ocelli realizes analog convolutions enabling low-precision ternary weight neural networks. Since the first layer’s convolution operations are the performance bottleneck of accelerators, Ocelli mitigates the overhead of analog buffers and analog-to-digital converters. Moreover, our design supports a zero-skipping scheme to further power reduction; (III) Ocelli exploits non-volatile magnetic RAMs to store CNN’s weights, which remarkably reduces the static power consumption; and finally, (IV) Ocelli has two modes, including sensing and processing. Once the object is detected, the architecture switches to the typical sensing mode to capture the image. Compared to the conventional pixels, it achieves an average 10% efficiency on its lane detection power consumption compared with existing edge detection algorithms. Moreover, considering different CNN workloads, our design shows more than 23% power efficiency over conventional designs, while it can achieve better accuracy.

show abstract

Section: Near/in-sensor Processing Backgroundmentioning

confidence: 99%

Ocelli: Efficient Processing-in-Pixel Array Enabling Edge Inference of Ternary Neural Networks

Tabrizchi

Angizi

Roohi

2022

JLPEA

View full text Add to dashboard Cite

show abstract

“…To mitigate this problem, prior works have proposed massively parallel in-memory [6], [18], in-sensor [5], [19], and in-pixel computing [2], [3], [7], [12], [13], [20], that aims to desegregate sensing, memory, and computation. Some insensor computing works implement analog computing in peripheral circuits for mapping AI algorithms [5], [19] but rely on serial kernel access that incurs significant energy and throughput bottlenecks.…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…Some insensor computing works implement analog computing in peripheral circuits for mapping AI algorithms [5], [19] but rely on serial kernel access that incurs significant energy and throughput bottlenecks. On the other hand, in-pixel computing approaches based on emerging technologies [2], [20] promise excellent energy and throughput improvements but are not compatible with the foundry-manufacturing of modern CMOS image sensor (CIS) platforms and hence are difficult to scale. Other solutions that are based on CMOS technology [3], [12], [13] have been limited to toy workloads, such as digit recognition, because they lack the support of multi-channel in-pixel convolutions.…”

Section: Introduction and Related Workmentioning

confidence: 99%

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

Datta¹,

Kundu²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Today's high resolution, high frame rate cameras in autonomous vehicles generate a large volume of data that needs to be transferred and processed by a downstream processor or machine learning (ML) accelerator to enable intelligent computing tasks, such as multi-object detection and tracking. The massive amount of data transfer incurs significant energy, latency, and bandwidth bottlenecks, which hinders real-time processing.To mitigate this problem, we propose an algorithm-hardware codesign framework called Processing-in-Pixel-in-Memory-based object Detection and Tracking (P 2 M-DeTrack). P 2 M-DeTrack is based on a custom faster R-CNN-based model that is distributed partly inside the pixel array (front-end) and partly in a separate FPGA/ASIC (back-end). The proposed front-end in-pixel processing down-samples the input feature maps significantly with judiciously optimized strided convolution and pooling. Compared to a conventional baseline design that transfers frames of RGB pixels to the back-end, the resulting P 2 M-DeTrack designs reduce the data bandwidth between sensor and back-end by up to 24×. The designs also reduce the sensor and total energy (obtained from in-house circuit simulations at Globalfoundries 22nm technology node) per frame by 5.7× and 1.14×, respectively. Lastly, they reduce the sensing and total frame latency by an estimated 1.7× and 3×, respectively. We evaluate our approach on the multi-object object detection (tracking) task of the large-scale BDD100K dataset and observe only a 0.5% reduction in the mean average precision (0.8% reduction in the identification F1 score) compared to the state-of-the-art.

show abstract

“…In contrast, in-pixel processing solutions, such as [11][12][13][14][15] , aim to embed processing capabilities within the individual CIS pixels. Initial efforts have focused on in-pixel analog convolution operation 14,15 but many 11,[14][15][16] require the use of emerging non-volatile memories or 2D materials. Unfortunately, these technologies are not yet mature and thus not amenable to the existing foundry-manufacturing of CIS.…”

Section: Introductionmentioning

confidence: 99%

P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications

Datta

Kundu

Yin

et al. 2022

Preprint

View full text Add to dashboard Cite

The demand to process vast amounts of data generated from state-of-the-art high resolution cameras has motivated novel energy-efficient on-device AI solutions. Visual data in such cameras are usually captured in analog voltages by a sensor pixel array, and then converted to the digital domain for subsequent AI processing using analog-to-digital converters (ADC). Recent research has tried to take advantage of massively parallel low-power analog/digital computing in the form of near- and in-sensor processing, in which the AI computation is performed partly in the periphery of the pixel array and partly in a separate on-board CPU/accelerator. Unfortunately, high-resolution input images still need to be streamed between the camera and the AI processing unit, frame by frame, causing energy, bandwidth, and security bottlenecks. To mitigate this problem, we propose a novel Processing-in-Pixel-in-memory (P2M) paradigm, that customizes the pixel array by adding support for analog multi-channel, multi-bit convolution, batch normalization, and ReLU (Rectified Linear Units). Our solution includes a holistic algorithm-circuit co-design approach and the resulting P2M paradigm can be used as a drop-in replacement for embedding memory-intensive first few layers of convolutional neural network (CNN) models within foundry-manufacturable CMOS image sensor platforms. Our experimental results indicate that P2M reduces data transfer bandwidth from sensors and analog to digital conversions by ~21x, and the energy-delay product (EDP) incurred in processing a MobileNetV2 model on a TinyML use case for visual wake words dataset (VWW) by up to ~11x compared to standard near-processing or in-sensor implementations, without any significant drop in test accuracy.

show abstract

PISA: A Binary-Weight Processing-In-Sensor Accelerator for Edge Image Processing

Cited by 5 publications

References 40 publications

Ocelli: Efficient Processing-in-Pixel Array Enabling Edge Inference of Ternary Neural Networks

Ocelli: Efficient Processing-in-Pixel Array Enabling Edge Inference of Ternary Neural Networks

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications

Contact Info

Product

Resources

About