Hardware for machine learning: Challenges and opportunities

Sze, Vivienne; Chen, Yu‐Hsin; Einer, Joel; Suleiman, Amr; Zhang, Zhengdong

doi:10.1109/cicc.2017.7993626

Cited by 198 publications

(134 citation statements)

References 60 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…There exists a severe contradiction between the complex model and the limited computational resources. Although at present, a large amount of dedicated hardware emerges for deep learning [16,17,18,19,20], providing efficient vector operations to enable fast convolution in forward inference, From the aspect of explainable machine learning, we can summarize that some filters are playing a similar role in the model, especially when the model size is large. So it is reasonable to prune some useless filters or reduce their precision to lower bits.…”

Section: Introductionmentioning

confidence: 99%

Binary neural networks: A survey

Qin

Gong

Liu

et al. 2020

Pattern Recognition

389

171

View full text Add to dashboard Cite

The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices.However, the binarization inevitably causes severe information loss, and even worse, its discontinuity brings difficulty to the optimization of the deep network.To address these issues, a variety of algorithms have been proposed, and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these algorithms, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error. We also investigate other practical aspects of binary neural networks such as the hardware-friendly design and the training tricks. Then, we give the evaluation and discussions on different tasks, including image classification, object detection and semantic segmentation. Finally, the challenges that may be faced in future research are prospected.the heavy computation and storage still inevitably limit the applications of the deep CNNs in practice. Besides, due to the huge model parameter space, the prediction of the neural networks is usually viewed as a black-box, which brings great challenges to the interpretability of CNNs. Some works like [21,22,23] empirically explore the function of each layer in the network. They visualize the feature maps extracted by different filters and view each filter as a visual unit focusing on different visual components.of the ResNet-50 [28], and meanwhile save more than 75% of parameters and 50% computational time. In the literature, approaches for compressing the deep networks can be classified into five categories: parameter pruning [26,29,30,31], parameter quantizing [32,33,34,35,36,37,38,39,40,41], low-rank parameter factorization [42,43,44,45,46], transferred/compact convolutional filters [47,48,49,50], and knowledge distillation [51,52,53,54,55,56]. The parameter pruning and quantizing mainly focus on eliminating the redundancy in the model parameters respectively by removing the redundant/uncritical ones or compressing the parameter space (e.g. , from the floating-point weights to the integer ones). Low-rank factorization applies the matrix/tensor decomposition techniques to estimate the informative parameters using the proxy ones of small size. The compact convolutional filter based approaches rely on the carefullydesigned structural convolutional filters to reduce the storage and computation complexity. The knowledge distillation methods try to distill a more compact model to reproduce the output of a larger network.Among the existing network compression techniques, quantization based one serves as a promising and fast solution that yields highly compact models compared to their floating-point counterparts, by representing the network weights with very low precision. Along this direction, the most extreme quantization is binarization, the interest...

show abstract

Section: Introductionmentioning

confidence: 99%

Binary neural networks: A survey

Qin

Gong

Liu

et al. 2020

Pattern Recognition

389

171

View full text Add to dashboard Cite

show abstract

“…The proposed algorithm was written in C++, QVGA resolution frames (320×240 pixels) are processed and the maximum RAM usage is 1.6 GB. Following recent trends and suggestions of other authors, which claim a jointly software and hardware design, the results of the present work focus on the frame rate and power consumption of the algorithm, which are hardware dependent [13], [45].…”

Section: Resultsmentioning

confidence: 99%

“…Consumption is one of the main limiting factors for deep learning application on embedded IoT end-nodes. The opportunities that CNNs offer for image processing are undeniable but their practical use and expansion will highly depend on the hardware design and the development of hardwareoriented algorithms [45]. The energy consumption of CNNs is dominated by data movement instead of the computation itself [13].…”

Section: Power Consumptionmentioning

confidence: 99%

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications

Blanco-Filgueira

Garcia-Lesta

Fernández-Sanjurjo

et al. 2019

IEEE Internet Things J.

View full text Add to dashboard Cite

Compute and memory demands of state-of-the-art deep learning methods are still a shortcoming that must be addressed to make them useful at IoT end-nodes. In particular, recent results depict a hopeful prospect for image processing using Convolutional Neural Netwoks, CNNs, but the gap between software and hardware implementations is already considerable for IoT and mobile edge computing applications due to their high power consumption. This proposal performs low-power and real time deep learning-based multiple object visual tracking implemented on an NVIDIA Jetson TX2 development kit. It includes a camera and wireless connection capability and it is battery powered for mobile and outdoor applications. A collection of representative sequences captured with the onboard camera, dETRUSC video dataset, is used to exemplify the performance of the proposed algorithm and to facilitate benchmarking. The results in terms of power consumption and frame rate demonstrate the feasibility of deep learning algorithms on embedded platforms although more effort to joint algorithm and hardware design of CNNs is needed.

show abstract

“…• Improvement in learning efficiency: Efficiency of learning could include some aspects such as the number of required parameters, memory or storage requirements, computational time and training convergence rate. Obviously, fewer but optimal parameters and lower memory requirements are desirable in deploying such algorithms on conventional devices such as mobile phones and PCs [59].…”

Section: Joint Tasks Via Multitask Learningmentioning

confidence: 99%

Image-to-Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography

Eslami

Tabarestani

Albarqouni

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widelyused for the diagnosis, screening and treatment follow up of diseases related to lungs and heart. The literature in this field of research reports many interesting studies dealing with the challenging tasks of bone suppression and organ segmentation but performed separately, limiting any learning that comes with the consolidation of parameters that could optimize both processes. Although image processing could facilitate computer aided diagnosis, machine learning seems more amenable in dealing with the many parameters one would have to contend with to yield an near optimal classification or decision-making process. This study, and for the first time, introduces a multitask deep learning model that generates simultaneously the bonesuppressed image and the organ segmented image, minimizing as a consequence the number of parameters the model has to deal with and optimizing the processing time as well; while at the same time exploiting the interplay in these parameters so as to benefit the performance of both tasks. The design architecture of this model, which relies on a conditional generative adversarial network, reveals the process on how we managed to modify the well-established pix2pix network to fit the need for multitasking and hence extending the standard image-to-image network to the new image-to-images architecture. Dilated convolutions are also used to improve the results through a more effective receptive field assessment. A comparison of the proposed approach to state-of-the-art algorithms is provided to gauge the merits of the proposed approach.

show abstract

Hardware for machine learning: Challenges and opportunities

Cited by 198 publications

References 60 publications

Binary neural networks: A survey

Binary neural networks: A survey

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications

Image-to-Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography

Contact Info

Product

Resources

About