Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition

Panda, Priyadarshini; Sengupta, Abhronil; Roy, Kaushik

doi:10.3850/9783981537079_0819

Cited by 118 publications

(86 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During test time, input instance is passed through each stage to produce a class label. Evaluation of the CDL methodology on the MNIST dataset demonstrates 1.91x reduction in average operations per input [30]. In addition to energy efficiency, we also observe that the CDL network outperforms the baseline DLNN in terms of classification accuracy (97.5% for DLNN and for 98.9% CDL).…”

Section: Conditional Deep Learningmentioning

confidence: 83%

“…It is well-known that the convolutional layers (CNN layers) of a DLNN, interpreted as visual layers, learn a hierarchy of features which transition from general (similar to Gabor filters and color blobs [28]) to specific, as we go deeper into the network [29]. We can utilize the generic-to-specific transition in the learnt features [30]. One way of achieving such a goal is to add a linear network of output neurons for each convolutional layer and monitor the output of the linear network to conditionally activate the deeper layers.…”

Section: Conditional Deep Learningmentioning

confidence: 99%

See 1 more Smart Citation

Invited - Cross-layer approximations for neuromorphic computing: from devices to circuits and systems

Panda

Sengupta

Sarwar

et al. 2016

Proceedings of the 53rd Annual Design Automation Conference

Self Cite

View full text Add to dashboard Cite

Neuromorphic algorithms are being increasingly deployed across the entire computing spectrum from data centers to mobile and wearable devices to solve problems involving recognition, analytics, search and inference. For example, large-scale artificial neural networks (popularly called deep learning) now represent the stateof-the art in a wide and ever-increasing range of video/image/audio/text recognition problems. However, the growth in data sets and network complexities have led to deep learning becoming one of the most challenging workloads across the computing spectrum. We posit that approximate computing can play a key role in the quest for energy-efficient neuromorphic systems. We show how the principles of approximate computing can be applied to the design of neuromorphic systems at various layers of the computing stack. At the algorithm level, we present techniques to significantly scale down the computational requirements of a neural network with minimal impact on its accuracy. At the circuit level, we show how approximate logic and memory can be used to implement neurons and synapses in an energy-efficient manner, while still meeting accuracy requirements. A fundamental limitation to the efficiency of neuromorphic computing in traditional implementations (software and custom hardware alike) is the mismatch between neuromorphic algorithms and the underlying computing models such as von Neumann architecture and Boolean logic. To overcome this limitation, we describe how emerging spintronic devices can offer highly efficient, approximate realization of the building blocks of neuromorphic computing systems. 2016, Austin, TX, USA © 2016 ACM. ISBN 978-1-4503-4236-0/16/06…$15.00

show abstract

Section: Conditional Deep Learningmentioning

confidence: 83%

Section: Conditional Deep Learningmentioning

confidence: 99%

Invited - Cross-layer approximations for neuromorphic computing: from devices to circuits and systems

Panda

Sengupta

Sarwar

et al. 2016

Proceedings of the 53rd Annual Design Automation Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…Future work will include enhancing the denoised powers-of-two networks with other complexity reduction techniques such as network pruning or conditional execution [34][35][36].…”

Section: Resultsmentioning

confidence: 99%

Robust and Computationally-Efficient Anomaly Detection Using Powers-Of-Two Networks

Muneeb

Koyuncu

Keshtkarjahromi

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Robust and computationally efficient anomaly detection in videos is a problem in video surveillance systems. We propose a technique to increase robustness and reduce computational complexity in a Convolutional Neural Network (CNN) based anomaly detector that utilizes the optical flow information of video data. We reduce the complexity of the network by denoising the intermediate layer outputs of the CNN and by using powers-of-two weights, which replaces the computationally expensive multiplication operations with bit-shift operations. Denoising operation during inference forces small valued intermediate layer outputs to zero. The number of zeros in the network significantly increases as a result of denoising, we can implement the CNN about 10% faster than a comparable network while detecting all the anomalies in the testing set. It turns out that denoising operation also provides robustness because the contribution of small intermediate values to the final result is negligible. During training we also generate motion vector images by a Generative Adversarial Network (GAN) to improve the robustness of the overall system. We experimentally observe that the resulting system is robust to background motion.

show abstract

“…The structure of the network is showed in Fig.1 [13]. There are three convolution layers which followed by liner classifiers.…”

Section: The Cdln Networkmentioning

confidence: 99%

“…In 2016, Panda p et al [13] propose a Conditional Deep Learning Network(CDLN) which adding extra linear classifiers behind the convolution layers. Through monitoring the output of the liner classifiers the ones that are easy to classify is classified in advance and exit the network for fast inference.…”

Section: Convolutional Neural Networkmentioning

confidence: 99%

Compression of Conditional Deep Learning Network for Fast and Low Power Mobile Applications

Li¹,

Zhang²,

Wang³

2017

Proceedings of the 2nd International Conference on Mechatronics Engineering and Information Technology (ICMEIT 2017)

View full text Add to dashboard Cite

Abstract. CDLN(Conditional Deep Learning Network)is a structure of convolution neural network with multiple classifiers. CDLN could improve the speed for the task of classification while the module of the network is still too large for mobile devices. To address this issue, a method for compressing CDLN, which is named one-shot whole network compression scheme. In the experiments, the module size and time cost are significantly reduced while the accuracy of the network losses a little. Convolutional Neural NetworkConvolutional Neural Network (CNN), which is widely applied in computer vision, is a representative method of deep learning due to its excellent learning ability for high dimensional data feature [1]. Recent years, with the emergence of related learning techniques, optimization techniques and hardware technology, convolution neural network has explosively developed[2]. ImageNet Large Scale Visual Recognition Challenge (ILSVRC) is a standard challenge for large-scale recognition. CNN has been widely used in classification activities of ImageNet and has achieved excellent classification results [3]. From the 8-layer AlexNet[4] to the 19-layer VGGNet[5] and the 152-layer ResNet[6], CNN is going deeper and deeper and the top-5 error reduces from 15.3% to 6.8% and 3.57%. However, the cost time and energy of forward propagation when training the network increase drastically [7]. For example, the running time of VGGNet is 20 times of AlexNet when performing classification tasks in the same dataset and experimental condition [8]. In addition, engineers and developers usually need to take time cost in concern in the context of industrial and commercial applications [9]. For instance, the online search engines need to response rapidly, and the cloud service needs to deal with thousands of pictures per second. Otherwise, the applications for scene recognition on smart phones and portable devices, which are lack of powerful ability for computing, need to response quickly.In 2011, Vanhoucke et al [10] research the method of the code optimization to speed up execution of CNN to reduce the runtime of the network.In 2013, Mathieu et al [11] convolute the convolution value as the dot product of the Fourier domain, when repeat the use of the same transform feature map, for the aid of reducing the runtime.In 2015, Kim et al [12] apply Tucker decomposition to extract the shared information between the convolution layer and the execution rank selection. This method reduces the number of network parameters for fast inference at the expense of the accuracy.In 2016, Panda p et al [13] propose a Conditional Deep Learning Network(CDLN) which adding extra linear classifiers behind the convolution layers. Through monitoring the output of the liner classifiers the ones that are easy to classify is classified in advance and exit the network for fast inference. The structure of the network is modified in CDLN which is a novel way.In this paper, a method for compressing the module of the network is applied to compress the CDLN for fas...

show abstract

Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition

Cited by 118 publications

References 15 publications

Invited - Cross-layer approximations for neuromorphic computing: from devices to circuits and systems

Invited - Cross-layer approximations for neuromorphic computing: from devices to circuits and systems

Robust and Computationally-Efficient Anomaly Detection Using Powers-Of-Two Networks

Compression of Conditional Deep Learning Network for Fast and Low Power Mobile Applications

Contact Info

Product

Resources

About