A processing-in-pixel-in-memory paradigm for resource-constrained TinyML applications

Datta, Gourav; Kundu, Souvik; Yin, Zihan; Lakkireddy, Ravi Teja; Mathai, Joe; Jacob, Ajey; Beerel, Peter A.; Jaiswal, Akhilesh

doi:10.1038/s41598-022-17934-1

Cited by 16 publications

(4 citation statements)

References 52 publications

(79 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…defined such that P(0) = −P r ; P sat is the saturation polarization, with P sat ≥ P r . Following Jiang et al [24], asymmetry in the positive and negative branches of the switching loop could be accounted for by assuming different E c values for the positive and negative polarity, replacing E c , k, and P o f f with E c,± , k ± , and P o f f ,± in Equations ( 1) and (2).…”

Section: Ftj: Device Characteristics and Modelingmentioning

confidence: 99%

See 1 more Smart Citation

A Low-Power Ternary Adder Using Ferroelectric Tunnel Junctions

et al. 2023

View full text Add to dashboard Cite

Computing systems are becoming more and more power-constrained due to unconventional computing requirements like computing on the edge, in-sensor, or simply an insufficient battery. Emerging Non-Volatile Memories are explored to build low-power computing circuits, and adders are one among them. In this work, we propose a low-power adder using a Ferroelectric Tunnel Junction (FTJ). FTJs are two-terminal devices where the data is stored in the polarization state of the device. An FTJ-based majority gate is proposed, which uses a current-mode sensing technique to evaluate the majority of the inputs. By conditionally selecting between the majority and its complement, an XOR operation is implemented, thereby achieving full-adder functionality. Since FTJ-based majority operation is slow, a ternary adder architecture is used to compensate for the speed loss. The ternary adder proposed by us has two stages of full adder and requires O(1) time for n-bit addition. The proposed adder is verified using a simulation in CMOS 130 nm technology. A 32-bit addition can be achieved in 100 μs and consumes 0.78 pJ, which is very power efficient (7.8 nW). The proposed adder can be used in applications where power consumption is crucial, and speed is not a strict requirement.

show abstract

Section: Ftj: Device Characteristics and Modelingmentioning

confidence: 99%

“…Computing systems are becoming more and more power-constrained due to the need for computing capability on the edge (edge computing [1]), battery-operated sensors (in-sensor processing [2]), and other such applications. Consequently, there is a great need for low-power circuits.…”

Section: Introductionmentioning

confidence: 99%

A Low-Power Ternary Adder Using Ferroelectric Tunnel Junctions

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This has always been a problem, especially in environments with limited resources and limited access to cutting-edge medical equipment. However, recent technological advancements, particularly in the areas of machine learning (ML) and deep learning (DL) [2] [3], have opened up new opportunities for the creation of sophisticated monitoring systems that are able to function effectively even on devices with limited resources. TinyML transforms classification processes on IoT devices with limited resources by installing lightweight machine learning models directly on these devices, doing away with the requirement for continuous data transfer to centralised servers.…”

Section: Introductionmentioning

confidence: 99%

ETL-FEXIC Model For Secured Heart Rate Abnormality Healthcare Framework

Arthi R

2024

jes

View full text Add to dashboard Cite

In traditional methods, it is critical for an effective continuous pulse monitor for humans prone to heart rate abnormalities. This paper proposes a secured heartrate abnormality detector which continuously monitors human pulse rate and SpO2 level. The current studies proposes that machine learning (ML) models performs well in classification; also, TinyML model shows better performance for data from resource constrained IoT devices. Hence, the research first analyses abnormal heart rate detection and spam data identification using standard ML algorithms such as SVM, Random Forest, Decision Tree, and TinyML. Though ML models are superior in classification, deep learning approaches outperforms them in feature learning. Hence, our proposed framework combines the merits of both ML and DL models. In our approach, the generated healthcare dataset is fed to DL models such as ANN, and autoencoder and also to SHAP XAI (eXplainable Artificial Intelligence) for feature extraction and learning. These learnt features are fed to ML models for classification. In this experiment, the proposed ETL-FEXIC (Enhanced Tiny Machine Learning with Automated Feature Extraction) outperforms the other ML models where the extracted features from XAI is fed to optimized TinyML classification model.

show abstract

“…Furthermore, Xu et al ( 2021 ) performs classification tasks on the MNIST dataset by generating the in-pixel MAC results of the first BNN layer and exhibits 17.3 TOPS/W energy efficiency. In addition, a processing-in-pixel-in-memory paradigm for CIS reported an 11× energy-delay product (EDP) improvement on the Visual Wake Words (VWW) dataset (Datta et al, 2022c ). Follow-up works by the same authors have demonstrated 5.26 × and 3.14 × reduction in energy consumption on hyperspectral image recognition (Datta et al, 2022e ) and multi-object tracking in the wild (Datta et al, 2022d ), respectively.…”

Section: Introductionmentioning

confidence: 99%

Neuromorphic-P2M: processing-in-pixel-in-memory paradigm for neuromorphic image sensors

Kaiser

Datta

Wang

et al. 2023

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources. Hence, researchers have been exploring different energy-efficient solutions such as near-sensor, in-sensor, and in-pixel processing, bringing the computation closer to the sensor. In particular, in-pixel processing embeds the computation capabilities inside the pixel array and achieves high energy efficiency by generating low-level features instead of the raw data stream from CMOS image sensors. Many different in-pixel processing techniques and approaches have been demonstrated on conventional frame-based CMOS imagers; however, the processing-in-pixel approach for neuromorphic vision sensors has not been explored so far. In this work, for the first time, we propose an asynchronous non-von-Neumann analog processing-in-pixel paradigm to perform convolution operations by integrating in-situ multi-bit multi-channel convolution inside the pixel array performing analog multiply and accumulate (MAC) operations that consume significantly less energy than their digital MAC alternative. To make this approach viable, we incorporate the circuit's non-ideality, leakage, and process variations into a novel hardware-algorithm co-design framework that leverages extensive HSpice simulations of our proposed circuit using the GF22nm FD-SOI technology node. We verified our framework on state-of-the-art neuromorphic vision sensor datasets and show that our solution consumes ~2× lower backend-processor energy while maintaining almost similar front-end (sensor) energy on the IBM DVS128-Gesture dataset than the state-of-the-art while maintaining a high test accuracy of 88.36%.

show abstract

A processing-in-pixel-in-memory paradigm for resource-constrained TinyML applications

Cited by 16 publications

References 52 publications

A Low-Power Ternary Adder Using Ferroelectric Tunnel Junctions

A Low-Power Ternary Adder Using Ferroelectric Tunnel Junctions

ETL-FEXIC Model For Secured Heart Rate Abnormality Healthcare Framework

Neuromorphic-P2M: processing-in-pixel-in-memory paradigm for neuromorphic image sensors

Contact Info

Product

Resources

About