Dynamic Temperature Management of Near-Sensor Processing for Energy-Efficient High-Fidelity Imaging

Kodukula, Venkatesh; Katrawala, Saad; Jones, Britton; Wu, Carole Jean; LiKamWa, Robert

doi:10.3390/s21030926

Cited by 14 publications

(9 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is important to note that the energy metric presented in this work is based on the assumption that the front-end and back-end chips are closely located on the same printedcircuit board [15]. In general, the front-end and back-end sensors could be separated by large distances, necessitating long energy-expensive wired or wireless data transfer (which is common for the case of sensor-fusion and swarm intelligence applications).…”

Section: Conclusion and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

Datta¹,

Kundu²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Today's high resolution, high frame rate cameras in autonomous vehicles generate a large volume of data that needs to be transferred and processed by a downstream processor or machine learning (ML) accelerator to enable intelligent computing tasks, such as multi-object detection and tracking. The massive amount of data transfer incurs significant energy, latency, and bandwidth bottlenecks, which hinders real-time processing.To mitigate this problem, we propose an algorithm-hardware codesign framework called Processing-in-Pixel-in-Memory-based object Detection and Tracking (P 2 M-DeTrack). P 2 M-DeTrack is based on a custom faster R-CNN-based model that is distributed partly inside the pixel array (front-end) and partly in a separate FPGA/ASIC (back-end). The proposed front-end in-pixel processing down-samples the input feature maps significantly with judiciously optimized strided convolution and pooling. Compared to a conventional baseline design that transfers frames of RGB pixels to the back-end, the resulting P 2 M-DeTrack designs reduce the data bandwidth between sensor and back-end by up to 24×. The designs also reduce the sensor and total energy (obtained from in-house circuit simulations at Globalfoundries 22nm technology node) per frame by 5.7× and 1.14×, respectively. Lastly, they reduce the sensing and total frame latency by an estimated 1.7× and 3×, respectively. We evaluate our approach on the multi-object object detection (tracking) task of the large-scale BDD100K dataset and observe only a 0.5% reduction in the mean average precision (0.8% reduction in the identification F1 score) compared to the state-of-the-art.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

“…E com is computed from e com as shown in Eq. ( 4), and its value shown in Table IV is obtained from [15]. On the other hand, E soc in primarily dominated by multiply-and-add (MAdd) energy of the models used for detection (E mac ).…”

Section: E Reduction In Energy Consumptionmentioning

confidence: 99%

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

Datta¹,

Kundu²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To address the compute-efficiency, latency, and throughput bottlenecks of 2D computer vision algorithms, recent research have proposed several processing-near-sensor (PNS) [12,13], processing-in-sensor (PIS) [14], and PIP solutions [15,16]. PNS approaches incorporate a dedicated machine learning (ML) accelerator chip on the same printed circuit board [12], or 2.5D/3D stacked with a pixel chip [13]. PIS approaches, in contrast, leverage parallel analog computing in the peripheral circuits of a memory array [14].…”

Section: Energy-efficient On-device Visionmentioning

confidence: 99%

“…The FLOPs count is computed as the total number of multiply-and-accumulate (MAC) operations in the convolutional and linear layers, similar to [24,25,26], while the peak memory is evaluated using the same convention as [27]. The total energy is computed as the sum of the image sensor energy, the sensor-to-SoC communication energy obtained from [12], and the energy incurred in processing the CNN layers. Note that the sensor energy is the sum of the pixel array energy 2 that is obtained from our circuit simulations and the ADC energy that is obtained from [28].…”

Section: Analysis Of Energy-efficiencymentioning

confidence: 99%

Toward Efficient Hyperspectral Image Processing inside Camera Pixels

Datta¹,

Yin²,

Jacob³

et al. 2022

Preprint

View full text Add to dashboard Cite

Hyperspectral cameras generate a large amount of data due to the presence of hundreds of spectral bands as opposed to only three channels (red, green, and blue) in traditional cameras. This requires a significant amount of data transmission between the hyperspectral image sensor and a processor used to classify/detect/track the images, frame by frame, expending high energy and causing bandwidth and security bottlenecks. To mitigate this problem, we propose a form of processing-in-pixel (PIP) that leverages advanced CMOS technologies to enable the pixel array to perform a wide range of complex operations required by the modern convolutional neural networks (CNN) for hyperspectral image recognition (HSI). Consequently, our PIP-optimized custom CNN layers effectively compress the input data, significantly reducing the bandwidth required to transmit the data downstream to the HSI processing unit. This reduces the average energy consumption associated with pixel array of cameras and the CNN processing unit by 25.06× and 3.90× respectively, compared to existing hardware implementations. Our custom models yield average test accuracies within 0.56% of the baseline models for the standard HSI benchmarks.

show abstract

“…For all other uses, contact the owner/author(s). MobiSys '22, June 25-July analog signal chain [1,5] as the fundamental bottleneck for energyefficiency. Specifically, analog circuits alone can consume anywhere from 33% to 85% of camera power.…”

Section: Introductionmentioning

confidence: 99%

Adaptive voltage scaling to balance energy savings and image quality in cameras

Kodukula

Manetta

LiKamWa

2022

Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services

Self Cite

View full text Add to dashboard Cite

Energy-efficient visual sensing is extremely important to enable battery powered mobile and IoT applications. While several scheduling techniques have been proposed to save the digital power of sensing, the analog power [1] of capturing an image still remains a daunting barrier for a camera's energy-efficiency. To that end, we characterize the power and performance implications of analog voltage scaling on off-the-shelf image sensors. Our characterization reveals that while reducing the analog voltage supplied to image sensor helps promote sensor power efficiency, it also impairs imaging fidelity, specifically by making images brighter and noisier. Furthermore, we find that brighter and noisier images situationally affect the task accuracy of vision applications. In this poster, we propose an investigation towards a system that adaptively scales analog voltage to optimize sensor energy, while respecting the fidelity needs of visual tasks. CCS CONCEPTS• Computer systems organization → Reconfigurable computing.

show abstract

Dynamic Temperature Management of Near-Sensor Processing for Energy-Efficient High-Fidelity Imaging

Cited by 14 publications

References 37 publications

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

Toward Efficient Hyperspectral Image Processing inside Camera Pixels

Adaptive voltage scaling to balance energy savings and image quality in cameras

Contact Info

Product

Resources

About