daBNN

Zhang, Jianhao; Pan, Yingwei; Yao, Ting; Zhao, He; Mei, Tao

doi:10.1145/3343031.3350534

Cited by 46 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are some inference frameworks for BNN, some of them are open-source software frameworks like BMXNet [222], BMXNet 2 [223], daBNN [224], Riptide [225], and Larq [226], and the others are not available to the public like BitStream [227]. This section discusses only the open-source software frameworks.…”

Section: ) Other Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Literature Review on Binary Neural Networks

et al. 2023

View full text Add to dashboard Cite

This paper presents an extensive literature review on Binary Neural Network (BNN). BNN utilizes binary weights and activation function parameters to substitute the full-precision values. In digital implementations, BNN replaces the complex calculations of Convolutional Neural Networks (CNNs) with simple bitwise operations. BNN optimizes large computation and memory storage requirements, which leads to less area and power consumption compared to full-precision models. Although there are many advantages of BNN, the binarization process has a significant impact on the performance and accuracy of the generated models. To reflect the state-of-the-art in BNN and explore how to develop and improve BNNbased models, we conduct a systematic literature review on BNN with data extracted from 239 research studies. Our review discusses various BNN architectures and the optimization approaches developed to improve their performance. There are three main research directions in BNN: accuracy optimization, compression optimization, and acceleration optimization. The accuracy optimization approaches include quantization error reduction, special regularization, gradient error minimization, and network structure. The compression optimization approaches combine fractional BNN and pruning. The acceleration optimization approaches comprise computing in-memory, FPGA-based implementations, and ASIC-based implementations. At the end of our review, we present a comprehensive analysis of BNN applications and their evaluation metrics. Also, we shed some light on the most common BNN challenges and the future research trends of BNN.

show abstract

Section: ) Other Frameworkmentioning

confidence: 99%

“…daBNN [224] is a fast BNN inference framework for ARM devices, released under the BSD license. daBNN utilized the following methods to speed the inference: upgraded bitpacking scheme, direct binary convolution, and novel memory layout to decrease memory access.…”

Section: ) Other Frameworkmentioning

confidence: 99%

A Systematic Literature Review on Binary Neural Networks

et al. 2023

View full text Add to dashboard Cite

show abstract

“…BitFlow provides 1.8× speedup compared to naïve binary convolution. In this line, DaBnn is an efficient inference framework for binary neural networks on mobile platforms powered by ARM processors [50]. Our work is the first one studying the efficiency of low-bits quantized neural networks on CPU.…”

Section: Related Workmentioning

confidence: 99%

Distilled Neural Networks for Efficient Learning to Rank

Nardini

Cosimo

Trani

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Quantization and pruning are known to be two effective Deep Neural Networks model compression methods. In this paper, we propose Automatic Prune Binarization (APB), a novel compression technique combining quantization with pruning. APB enhances the representational capability of binary networks using a few full-precision weights. Our technique jointly maximizes the accuracy of the network while minimizing its memory impact by deciding whether each weight should be binarized or kept in full precision. We show how to efficiently perform a forward pass through layers compressed using APB by decomposing it into a binary and a sparse-dense matrix multiplication. Moreover, we design two novel efficient algorithms for extremely quantized matrix multiplication on CPU, leveraging highly efficient bitwise operations. The proposed algorithms are 6.9× and 1.5× faster than available state-of-the-art solutions. We perform an extensive evaluation of APB on two widely adopted model compression datasets, namely CIFAR-10 and ImageNet. APB shows to deliver better accuracy/memory trade-off compared to state-of-the-art methods based on i) quantization, ii) pruning, and iii) combination of pruning and quantization. APB outperforms quantization also in the accuracy/efficiency trade-off, being up to 2× faster than the 2-bits quantized model with no loss in accuracy.

show abstract

“…BNNs are suitable for working in limited-resource devices, improving the inference speed. However, in contraposition, BNNs may lose accuracy and precision, in particular when both activation and weights are binary [54][55][56]. DEXiRE employs binary neural networks to discretize only the neuron activations, based on Hypothesis 1, identifying active neurons, and making easier the induction of Boolean functions [57].…”

Section: Underlying Rational Designmentioning

confidence: 99%

A DEXiRE for Extracting Propositional Rules from Neural Networks via Binarization

et al. 2022

View full text Add to dashboard Cite

Background: Despite the advancement in eXplainable Artificial Intelligence, the explanations provided by model-agnostic predictors still call for improvements (i.e., lack of accurate descriptions of predictors’ behaviors). Contribution: We present a tool for Deep Explanations and Rule Extraction (DEXiRE) to approximate rules for Deep Learning models with any number of hidden layers. Methodology: DEXiRE proposes the binarization of neural networks to induce Boolean functions in the hidden layers, generating as many intermediate rule sets. A rule set is inducted between the first hidden layer and the input layer. Finally, the complete rule set is obtained using inverse substitution on intermediate rule sets and first-layer rules. Statistical tests and satisfiability algorithms reduce the final rule set’s size and complexity (filtering redundant, inconsistent, and non-frequent rules). DEXiRE has been tested in binary and multiclass classifications with six datasets having different structures and models. Results: The performance is consistent (in terms of accuracy, fidelity, and rule length) with respect to the state-of-the-art rule extractors (i.e., ECLAIRE). Moreover, compared with ECLAIRE, DEXiRE has generated shorter rules (i.e., up to 74% fewer terms) and has shortened the execution time (improving up to 197% in the best-case scenario). Conclusions: DEXiRE can be applied for binary and multiclass classification of deep learning predictors with any number of hidden layers. Moreover, DEXiRE can identify the activation pattern per class and use it to reduce the search space for rule extractors (pruning irrelevant/redundant neurons)—shorter rules and execution times with respect to ECLAIRE.

show abstract

daBNN

Cited by 46 publications

References 9 publications

A Systematic Literature Review on Binary Neural Networks

A Systematic Literature Review on Binary Neural Networks

Distilled Neural Networks for Efficient Learning to Rank

A DEXiRE for Extracting Propositional Rules from Neural Networks via Binarization

Contact Info

Product

Resources

About