Diffy: a Déjà vu-Free Differential Deep Neural Network Accelerator

Mahmoud, Magdi S.; Siu, Kevin; Moshovos, Andreas

doi:10.1109/micro.2018.00020

Cited by 51 publications

(23 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classic techniques include using smaller, simplified models [35,52,67], model compression [15,30,37], quantization [36,[63][64][65], and using specialized hardware [14,27,48]. Recent developments have also leveraged the temporal/spatial redundancies in real-time frames to simplify computations [10,22,46,55,71].…”

Section: Related Workmentioning

confidence: 99%

Low-Latency Proactive Continuous Vision

Gan

Qiu

Chen

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

Continuous vision is the cornerstone of a diverse range of intelligent applications found on emerging computing platforms such as autonomous machines and Augmented Reality glasses. A critical issue in today's continuous vision systems is their long end-to-end frame latency, which significantly impacts the system agility and user experience. We find that the long latency is fundamentally caused by the serialized execution model of today's continuous vision pipeline, whose key stages, including sensing, imaging, and vision computations, execute sequentially, leading to long frame latency. This paper seeks to reduce the end-to-end latency of continuous vision tasks. Our key idea is a new proactive vision execution model that breaks the sequential execution of the vision pipeline. Specifically, we propose to allow the pipeline front-end (sensing and imaging) to predict future frames; the pipeline back-end (vision algorithms) then predictively operates on the future frames to reduce frame latency. While the proactive execution model is generally applicable to any vision systems, we demonstrate its effectiveness using an implementation on resource-constrained mobile Systemson-a-chips (SoC). Our system, PVF, incorporates two techniques to overcome key challenges that arise in deploying proactive vision in mobile systems: it enables multiple outstanding speculative frames by exploiting the hardware heterogeneities in mobile SoCs; it reduces the energy overhead of prediction by exploiting the error-resilient nature of vision algorithms. We show that PVF reduces the frame latency by up to 92% under the same energy. CCS Concepts • Computer systems organization → Heterogeneous (hybrid) systems; Special purpose systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Low-Latency Proactive Continuous Vision

Gan

Qiu

Chen

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

show abstract

“…Comparison. Table 7 compares the eCNN with two state-of-theart processors for computational imaging applications: IDEAL [42] for BM3D and Diffy [41] for CNN. Both of them can only support the HD30 specification and already require high-end DRAM settings, i.e.…”

Section: Ecnn Performancementioning

confidence: 99%

“…To compare power consumption, we list the reported numbers from [42] and [41] for IDEAL and Diffy on the right of Table 7. Note that these numbers cannot be used to determine superiority directly because they are highly related to technology nodes, implementation details, and deployed models/algorithms.…”

Section: Ecnn Performancementioning

confidence: 99%

eCNN

Huang

Ding

Wang

et al. 2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have recently demonstrated superior quality for computational imaging applications. Therefore, they have great potential to revolutionize the image pipelines on cameras and displays. However, it is difficult for conventional CNN accelerators to support ultra-high-resolution videos at the edge due to their considerable DRAM bandwidth and power consumption. Therefore, finding a further memory-and computation-efficient microarchitecture is crucial to speed up this coming revolution.In this paper, we approach this goal by considering the inference flow, network model, instruction set, and processor design jointly to optimize hardware performance and image quality. We apply a block-based inference flow which can eliminate all the DRAM bandwidth for feature maps and accordingly propose a hardwareoriented network model, ERNet, to optimize image quality based on hardware constraints. Then we devise a coarse-grained instruction set architecture, FBISA, to support power-hungry convolution by massive parallelism. Finally, we implement an embedded processor-eCNN-which accommodates to ERNet and FBISA with a flexible processing architecture. Layout results show that it can support high-quality ERNets for super-resolution and denoising at up to 4K Ultra-HD 30 fps while using only DDR-400 and consuming 6.94W on average. By comparison, the state-of-the-art Diffy uses dualchannel DDR3-2133 and consumes 54.3W to support lower-quality VDSR at Full HD 30 fps. Lastly, we will also present application examples of high-performance style transfer and object recognition to demonstrate the flexibility of eCNN.

show abstract

“…Fundamentally, motion-based relaxations fall within the realm of incremental computing, a general technique used in program analysis and optimization [51,54] and applies beyond the temporal and/or vision domain. Diffy [47] exploits the spatial similarity across pixels in the same frame to improve DNN efficiency. Riera et al [56] exploit repeated ifmap elements in speech recognition.…”

Section: Related Workmentioning

confidence: 99%

Asv

Whatmough

Zhu

2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Estimating depth from stereo vision cameras, i.e., "depth from stereo", is critical to emerging intelligent applications deployed in energy-and performance-constrained devices, such as augmented reality headsets and mobile autonomous robots. While existing stereo vision systems make trade-offs between accuracy, performance and energy-efficiency, we describe ASV, an accelerated stereo vision system that simultaneously improves both performance and energy-efficiency while achieving high accuracy.The key to ASV is to exploit unique characteristics inherent to stereo vision, and apply stereo-specific optimizations, both algorithmically and computationally. We make two contributions. Firstly, we propose a new stereo algorithm, invariant-based stereo matching (ISM), that achieves significant speedup while retaining high accuracy. The algorithm combines classic "hand-crafted" stereo algorithms with recent developments in Deep Neural Networks (DNNs), by leveraging the correspondence invariant unique to stereo vision systems. Secondly, we observe that the bottleneck of the ISM algorithm is the DNN inference, and in particular the deconvolution operations that introduce massive compute-inefficiencies. We propose a set of software optimizations that mitigate these inefficiencies. We show that with less than 0.5% hardware area overhead, these algorithmic and computational optimizations can be effectively integrated within a conventional DNN accelerator. Overall, ASV achieves 5× speedup and 85% energy saving with 0.02% accuracy loss compared to today's DNN-based stereo vision systems.

show abstract

Diffy: a Déjà vu-Free Differential Deep Neural Network Accelerator

Cited by 51 publications

References 38 publications

Low-Latency Proactive Continuous Vision

Low-Latency Proactive Continuous Vision

eCNN

Asv

Contact Info

Product

Resources

About