Recent Advances in Efficient Computation of Deep Convolutional Neural Networks

Cheng, Jian; Wang, Peisong; Li, Gang; Hu, Qinghao; Lu, Hanqing

doi:10.48550/arxiv.1802.00939

Cited by 5 publications

(7 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can be seen that the gain rate not only considers the information gain of the training samples in attribute B , but also considers the amount of information generated by the v branches generated by the v attribute values of attribute B [ 17 ]. So in the above example, due to the existence of extended parameters, its gain rate tends to be reasonable.…”

Section: Decision Tree Algorithm Before and After Improvement Based O...mentioning

confidence: 99%

MOOC and Flipped Classroom Task-Based English Teaching Model for Colleges and Universities Using Data Mining and Few-Shot Learning Technology

Gong

Zhou

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

As a revolutionary education model, the flipped classroom teaching model has unique advantages over the traditional education model. How to change the teaching method of flipped classroom into a teaching method suitable for college English courses is a problem. The goal of this research is to investigate how to use data mining and few-shot learning technology to investigate the impact of MOOC and flipped classroom task-based college English teaching modes. This work provides a data mining-based decision tree algorithm and examines the enhanced decision tree method. The experimental results of this study demonstrate that two students in each of the two groups believe that the English teaching mode, which is mostly taught by traditional teachers, is very favorable to a thorough understanding of basic knowledge, accounting for 4% of the total. There are 17 students that believe the new teaching methodology is really beneficial, accounting for 34% of the total. This mode was determined to be effective for in-depth knowledge of the basics by a total of 25 students. It can be seen that the flipped classroom model is more popular with students.

show abstract

Section: Decision Tree Algorithm Before and After Improvement Based O...mentioning

confidence: 99%

MOOC and Flipped Classroom Task-Based English Teaching Model for Colleges and Universities Using Data Mining and Few-Shot Learning Technology

Gong

Zhou

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Quantized neural networks has made significant progress with training (Courbariaux, Bengio, and David 2015;Han, Mao, and Dally 2015;Zhu et al 2016;Rastegari et al 2016;Mishra et al 2017;Zmora et al 2018;Cheng et al 2018;Krishnamoorthi 2018;Li, Dong, and Wang 2019). The research of post-training quantization is conducted for scenarios when training is not available (Krishnamoorthi 2018;Meller et al 2019;Banner, Nahshan, and Soudry 2019;Zhao et al 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Quantization is a promising method for creating more energy-efficient deep learning systems (Han, Mao, and Dally 2015;Hubara et al 2017;Zmora et al 2018;Cheng et al 2018). By approximating real-valued weights and activations using low-bit numbers, quantized neural networks (QNNs) trained with state-of-the-art algorithms (e.g., Courbariaux, Bengio, and David 2015;Rastegari et al 2016;Louizos et al 2018;Li, Dong, and Wang 2019) can be shown to perform similarly as their full-precision counterparts (e.g., Jung et al 2019;Li, Dong, and Wang 2019).…”

Section: Introductionmentioning

confidence: 99%

Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Liu

Zhou

et al. 2021

AAAI

View full text Add to dashboard Cite

We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number. Computationally, we construct the multipoint quantization with an efficient greedy selection procedure, and adaptively decides the number of low precision points on each quantized weight vector based on the error of its output. This allows us to achieve higher precision levels for important weights that greatly influence the outputs, yielding an ``effect of mixed precision'' but without physical mixed precision implementations (which requires specialized hardware accelerators). Empirically, our method can be implemented by common operands, bringing almost no memory and computation overhead. We show that our method outperforms a range of state-of-the-art methods on ImageNet classification and it can be generalized to more challenging tasks like PASCAL VOC object detection.

show abstract

“…Quantized neural networks has made significant progress with training (Courbariaux et al, 2015;Han et al, 2015;Zhu et al, 2016;Rastegari et al, 2016;Mishra et al, 2017;Zmora et al, 2018;Cheng et al, 2018;Krishnamoorthi, 2018;Li et al, 2019). The research of post-training quantization is conducted for scenarios when training is not available (Krishnamoorthi, 2018;Meller et al, 2019;Banner et al, 2019;Zhao et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Quantization is a promising method for creating more energy-efficient deep learning systems (Han et al, 2015;Hubara et al, 2017;Zmora et al, 2018;Cheng et al, 2018). By approximating real-valued weights and activations using low-bit numbers, quantized neural networks (QNNs) trained with state-of-the-art algorithms (e.g., Courbariaux et al, 2015;Rastegari et al, 2016;Louizos et al, 2018;Li et al, 2019) can be shown to perform similarly as their full-precision counterparts (e.g., Jung et al, 2019;Li et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Liu¹,

Ye²,

Zhou³

et al. 2020

Preprint

View full text Add to dashboard Cite

We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a fullprecision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number. Computationally, we construct the multipoint quantization with an efficient greedy selection procedure, and adaptively decides the number of low precision points on each quantized weight vector based on the error of its output. This allows us to achieve higher precision levels for important weights that greatly influence the outputs, yielding an "effect of mixed precision" but without physical mixed precision implementations (which requires specialized hardware accelerators (Wang et al., 2019)). Empirically, our method can be implemented by common operands, bringing almost no memory and computation overhead. We show that our method outperforms a range of state-of-the-art methods on ImageNet classification and it can be generalized to more challenging tasks like PASCAL VOC object detection.

show abstract

Recent Advances in Efficient Computation of Deep Convolutional Neural Networks

Cited by 5 publications

References 55 publications

MOOC and Flipped Classroom Task-Based English Teaching Model for Colleges and Universities Using Data Mining and Few-Shot Learning Technology

MOOC and Flipped Classroom Task-Based English Teaching Model for Colleges and Universities Using Data Mining and Few-Shot Learning Technology

Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Contact Info

Product

Resources

About