Exploring computation-communication tradeoffs in camera systems

Mazumdar, Amrita; Moreau, Thierry; Kim, Sung; Cowan, Meghan; Alaghi, Armin; Ceze, Luís; Oskin, Mark; Sathe, Visvesh

doi:10.1109/iiswc.2017.8167775

Cited by 11 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While this paper exploits temporal redundancy to avoid CNN layer computation, AMC also suggests opportunities for savings in the broader system. Future work can integrate camera sensors that avoid spending energy to capture redundant data [28,[62][63][64], and end-to-end visual applications can inform the system about which semantic changes are relevant for their task. A change-oriented visual system could exploit the motion vectors that hardware video codecs already produce, as recent work has done for super-resolution [26].…”

Section: Discussionmentioning

confidence: 99%

EVA²: Exploiting Temporal Redundancy in Live Computer Vision

Buckler

Bedoukian

Jayasuriya

et al. 2018

2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)

View full text Add to dashboard Cite

Hardware support for deep convolutional neural networks (CNNs) is critical to advanced computer vision in mobile and embedded devices. Current designs, however, accelerate generic CNNs; they do not exploit the unique characteristics of real-time vision. We propose to use the temporal redundancy in natural video to avoid unnecessary computation on most frames. A new algorithm, activation motion compensation, detects changes in the visual input and incrementally updates a previously-computed activation. The technique takes inspiration from video compression and applies well-known motion estimation techniques to adapt to visual changes. We use an adaptive key frame rate to control the trade-off between efficiency and vision quality as the input changes. We implement the technique in hardware as an extension to state-of-the-art CNN accelerator designs. The new unit reduces the average energy per frame by 54%, 62%, and 87% for three CNNs with less than 1% loss in vision accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

EVA²: Exploiting Temporal Redundancy in Live Computer Vision

Buckler

Bedoukian

Jayasuriya

et al. 2018

2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)

View full text Add to dashboard Cite

show abstract

“…The NPU in the Mesorasi architecture is a DNN accelerator specialized to point cloud processing. Mesorasi also extends beyond prior visual accelerators that deal with 2D data (images and videos) [32], [37], [38], [63], [64], [66] to 3D point clouds.…”

Section: Related Workmentioning

confidence: 99%

Mesorasi: Architecture Support for Point Cloud Analytics via Delayed-Aggregation

Tian

et al. 2020

Preprint

View full text Add to dashboard Cite

Point cloud analytics is poised to become a key workload on battery-powered embedded and mobile platforms in a wide range of emerging application domains, such as autonomous driving, robotics, and augmented reality, where efficiency is paramount. This paper proposes Mesorasi, an algorithm-architecture co-designed system that simultaneously improves the performance and energy efficiency of point cloud analytics while retaining its accuracy.Our extensive characterizations of state-of-the-art point cloud algorithms show that, while structurally reminiscent of convolutional neural networks (CNNs), point cloud algorithms exhibit inherent compute and memory inefficiencies due to the unique characteristics of point cloud data. We propose delayed-aggregation, a new algorithmic primitive for building efficient point cloud algorithms. Delayed-aggregation hides the performance bottlenecks and reduces the compute and memory redundancies by exploiting the approximately distributive property of key operations in point cloud algorithms. Delayed-aggregation let point cloud algorithms achieve 1.6× speedup and 51.1% energy reduction on a mobile GPU while retaining the accuracy (-0.9% loss to 1.2% gains). To maximize the algorithmic benefits, we propose minor extensions to contemporary CNN accelerators, which can be integrated into a mobile Systems-on-a-Chip (SoC) without modifying other SoC components. With additional hardware support, Mesorasi achieves up to 3.6× speedup.

show abstract

“…Stereo Vision Accelerators Recently, commercial mobile vision systems [77] have started integrating dedicated stereo accelerators, such as the Stereo Depth Block in the Movidius Enhanced Vision Accelerator Suite [5], and the Stereo & Optical Flow Engine (SOFE) in the Nvidia Xavier mobile SoC [9]. From publicly available details, these are fixed-functioned accelerators targeting classic stereo algorithms, similar to previous stereo vision accelerators [28,49,64,66,73]. In contrast, ASV combines the efficiency of classic stereo algorithms with the accuracy of stereo DNNs.…”

Section: Related Workmentioning

confidence: 99%

Asv

Whatmough

Zhu

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Estimating depth from stereo vision cameras, i.e., "depth from stereo", is critical to emerging intelligent applications deployed in energy-and performance-constrained devices, such as augmented reality headsets and mobile autonomous robots. While existing stereo vision systems make trade-offs between accuracy, performance and energy-efficiency, we describe ASV, an accelerated stereo vision system that simultaneously improves both performance and energy-efficiency while achieving high accuracy.The key to ASV is to exploit unique characteristics inherent to stereo vision, and apply stereo-specific optimizations, both algorithmically and computationally. We make two contributions. Firstly, we propose a new stereo algorithm, invariant-based stereo matching (ISM), that achieves significant speedup while retaining high accuracy. The algorithm combines classic "hand-crafted" stereo algorithms with recent developments in Deep Neural Networks (DNNs), by leveraging the correspondence invariant unique to stereo vision systems. Secondly, we observe that the bottleneck of the ISM algorithm is the DNN inference, and in particular the deconvolution operations that introduce massive compute-inefficiencies. We propose a set of software optimizations that mitigate these inefficiencies. We show that with less than 0.5% hardware area overhead, these algorithmic and computational optimizations can be effectively integrated within a conventional DNN accelerator. Overall, ASV achieves 5× speedup and 85% energy saving with 0.02% accuracy loss compared to today's DNN-based stereo vision systems.

show abstract

Exploring computation-communication tradeoffs in camera systems

Cited by 11 publications

References 33 publications

EVA²: Exploiting Temporal Redundancy in Live Computer Vision

EVA²: Exploiting Temporal Redundancy in Live Computer Vision

Mesorasi: Architecture Support for Point Cloud Analytics via Delayed-Aggregation

Asv

Contact Info

Product

Resources

About