RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision

LiKamWa, Robert; Hou, Yunhui; Gao, Yuan; Polansky, Mia; Lin, Zhouchen

doi:10.1109/isca.2016.31

Cited by 89 publications

(80 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, existing literature suggests that some PVT variations and noise in the circuit may not greatly impact application level accuracy for both MOSFETs and emerging devices (e.g. see [39,64]), thus individualized training would not be needed. Researchers have also investigated on-chip training given device variations (e.g., see [46,63]), and reasonable application level accuracy results are indeed obtained.…”

Section: Training With Actual I-v Characteristicsmentioning

confidence: 99%

A Mixed Signal Architecture for Convolutional Neural Networks

Lou

Pan²,

McGuinness

et al. 2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

Deep neural network (DNN) accelerators with improved energy and delay are desirable for meeting the requirements of hardware targeted for IoT and edge computing systems. Convolutional neural networks (CoNNs) belong to one of the most popular types of DNN architectures. is paper presents the design and evaluation of an accelerator for CoNNs. e system-level architecture is based on mixed-signal, cellular neural networks (CeNNs). Speci cally, we present (i) the implementation of di erent layers, including convolution, ReLU, and pooling, in a CoNN using CeNN, (ii) modi ed CoNN structures with CeNN-friendly layers to reduce computational overheads typically associated with a CoNN, (iii) a mixed-signal CeNN architecture that performs CoNN computations in the analog and mixed signal domain, and (iv) design space exploration that identi es what CeNN-based algorithm and architectural features fare best compared to existing algorithms and architectures when evaluated over common datasets -MNIST and CIFAR-10. Notably, the proposed approach can lead to 8.7× improvements in energy-delay product (EDP) per digit classi cation for the MNIST dataset at iso-accuracy when compared with the state-of-the-art DNN engine, while our approach could o er 4.3× improvements in EDP when compared to other network implementations for the CIFAR-10 dataset.When considering application-speci c hardware to support neural networks, it is important that said hardware can implement networks that can be extensible to a large class of networks, and solve a large collection of application-level problems. Deep neural networks (DNNs) represent a class of such networks and have demonstrated their strength in applications such as playing the game of Go [54], image and video analysis [32], target tracking [31], etc. In this paper, we use convolutional neural network (CoNN) as a case study for DNNs due to its general prevalence. CoNNs are computationally intensive, which could lead to high latency and energy for inference and even higher latency/energy for training. e focus of this paper is on developing a low energy/delay mixed-signal system based on cellular neural networks (CeNNs) for realizing CoNN.A Cellular Nonlinear/Neural Network (CeNN) is an analog computing architecture [11] that could be well suited for many information processing tasks. In a CeNN, identical processing units (called cells) process analog information in a concurrent manner. Interconnection between cells is typically local (i.e., nearest neighbor) and space-invariant. For spatio-temporal applications, CeNNs can o er vastly superior performance and power e ciency when compared to conventional von Neumann architectures [47,61]. Using "CeNNs for CoNN" allows the bulk of the computation associated with a CoNN to be performed in the analog domain. Sensed information could immediately be processed with no analog-to-digital conversion (ADC). Also, inference-based processing tasks can tolerate lower precision (e.g., Google's TPU employs 8-bit integer matrix multiplies [24]) typically associa...

show abstract

Section: Training With Actual I-v Characteristicsmentioning

confidence: 99%

A Mixed Signal Architecture for Convolutional Neural Networks

Lou

Pan²,

McGuinness

et al. 2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…However, existing NNADCs [5]- [7] often exhibit modest conversion resolution (4∼8-bit) and invariably rely on optimistic assumption of the RRAM precision (6∼12-bit), which is not well substantiated by measurement data from realistic RRAM fabrication process [8], [9]. This resolution limitation severely constrains the application of NNADCs in emerging multi-sensor systems that require high-resolution (>10-bit) A/D interfaces for feature extraction and near-sensor processing [1], [3], [4].…”

Section: B Nn-inspired Adcsmentioning

confidence: 99%

“…Many emerging applications have posed new challenges for the design of conventional analog-to-digital (A/D) converters (ADCs) [1]- [4]. For example, multi-sensor systems desire programmable nonlinear A/D quantization to maximize the extraction of useful features from the raw analog signal, instead of directly performing uniform quantization by conventional ADCs [3], [4].…”

Section: Introductionmentioning

confidence: 99%

Neural Network-Inspired Analog-to-Digital Conversion to Achieve Super-Resolution with Low-Precision RRAM Devices

Cao

Liu

Chakrabarti

et al. 2019

2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)

View full text Add to dashboard Cite

Recent works propose neural network-(NN-) inspired analog-to-digital converters (NNADCs) and demonstrate their great potentials in many emerging applications. These NNADCs often rely on resistive random-access memory (RRAM) devices to realize the NN operations and require high-precision RRAM cells (6∼12-bit) to achieve a moderate quantization resolution (4∼8-bit). Such optimistic assumption of RRAM resolution, however, is not supported by fabrication data of RRAM arrays in large-scale production process. In this paper, we propose an NN-inspired super-resolution ADC based on low-precision RRAM devices by taking the advantage of a co-design methodology that combines a pipelined hardware architecture with a custom NN training framework. Results obtained from SPICE simulations demonstrate that our method leads to robust design of a 14-bit super-resolution ADC using 3bit RRAM devices with improved power and speed performance and competitive figure-of-merits (FoMs). In addition to the linear uniform quantization, the proposed ADC can also support configurable high-resolution nonlinear quantization with high conversion speed and low conversion energy, enabling future intelligent analog-to-information interfaces for nearsensor analytics and processing.

show abstract

“…Computer Vision on Raw Sensor Data Diamond et al [60] and Buckler et al [44] both showed that CNN models can be effectively trained using raw image sensor data. Red-Eye [83] and ASP Vision [47] both move early CNN layer(s) into the camera sensor and compute using raw sensor data. This line of work is complementary to Euphrates in that our algorithm makes no assumption about which image format motion vectors are generated.…”

Section: Related Workmentioning

confidence: 99%

Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision

Zhu

Samajdar

Mattina

et al. 2018

2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)

View full text Add to dashboard Cite

Continuous computer vision (CV) tasks increasingly rely on convolutional neural networks (CNN). However, CNNs have massive compute demands that far exceed the performance and energy constraints of mobile devices. In this paper, we propose and develop an algorithm-architecture co-designed system, Euphrates, that simultaneously improves the energyefficiency and performance of continuous vision tasks.Our key observation is that changes in pixel data between consecutive frames represents visual motion. We first propose an algorithm that leverages this motion information to relax the number of expensive CNN inferences required by continuous vision applications. We co-design a mobile System-ona-Chip (SoC) architecture to maximize the efficiency of the new algorithm. The key to our architectural augmentation is to co-optimize different SoC IP blocks in the vision pipeline collectively. Specifically, we propose to expose the motion data that is naturally generated by the Image Signal Processor (ISP) early in the vision pipeline to the CNN engine. Measurement and synthesis results show that Euphrates achieves up to 66% SoC-level energy savings (4× for the vision computations), with only 1% accuracy loss.

show abstract

RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision

Cited by 89 publications

References 33 publications

A Mixed Signal Architecture for Convolutional Neural Networks

A Mixed Signal Architecture for Convolutional Neural Networks

Neural Network-Inspired Analog-to-Digital Conversion to Achieve Super-Resolution with Low-Precision RRAM Devices

Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision

Contact Info

Product

Resources

About