Larq: An Open-Source Library for Training Binarized Neural Networks

Geiger, Lukas; Team, Plumerai

doi:10.21105/joss.01746

Cited by 74 publications

(43 citation statements)

References 7 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we discuss experimental results of using HANNs for classifying synthetic and real datasets. Our implementation uses TensorFlow (Abadi et al, 2016) with the Larq (Geiger & Team, 2020) library for training neural networks with threshold activations. Note that Theorem 5.2 holds for ERM over HANNs, which is intractable in practice.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

VC dimension of partially quantized neural networks in the overparametrized regime

Wang¹,

Scott²

2021

Preprint

View full text Add to dashboard Cite

Vapnik-Chervonenkis (VC) theory has so far been unable to explain the small generalization error of overparametrized neural networks. Indeed, existing applications of VC theory to large networks obtain upper bounds on VC dimension that are proportional to the number of weights, and for a large class of networks, these upper bound are known to be tight. In this work, we focus on a class of partially quantized networks that we refer to as hyperplane arrangement neural networks (HANNs). Using a sample compression analysis, we show that HANNs can have VC dimension significantly smaller than the number of weights, while being highly expressive. In particular, empirical risk minimization over HANNs in the overparametrized regime achieves the minimax rate for classification with Lipschitz posterior class probability. We further demonstrate the expressivity of HANNs empirically. On a panel of 121 UCI datasets, overparametrized HANNs match the performance of state-of-the-art full-precision models.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Heuristic for coarse gradient of the threshold function. We use the SwishSign from the Larq library (Geiger & Team, 2020). Hyperparameters.…”

Section: B Training Detailsmentioning

confidence: 99%

VC dimension of partially quantized neural networks in the overparametrized regime

Wang¹,

Scott²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Note that in the first block we omitted the sign layer in order to improve the network's accuracy. 5 The model was trained for 300 epochs using the Larq library [17] and the Adam optimizer [36], achieving 90% accuracy. Fig.…”

Section: Discussionmentioning

confidence: 99%

An SMT-Based Approach for Verifying Binarized Neural Networks

Amir

Barrett

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep learning has emerged as an effective approach for creating modern software systems, with neural networks often surpassing hand-crafted systems. Unfortunately, neural networks are known to suffer from various safety and security issues. Formal verification is a promising avenue for tackling this difficulty, by formally certifying that networks are correct. We propose an SMT-based technique for verifying binarized neural networks — a popular kind of neural network, where some weights have been binarized in order to render the neural network more memory and energy efficient, and quicker to evaluate. One novelty of our technique is that it allows the verification of neural networks that include both binarized and non-binarized components. Neural network verification is computationally very difficult, and so we propose here various optimizations, integrated into our SMT procedure as deduction steps, as well as an approach for parallelizing verification queries. We implement our technique as an extension to the Marabou framework, and use it to evaluate the approach on popular binarized neural network architectures.

show abstract

“…Our BNN training is performed on an Intel i5-7400 CPU @ 3.000 GHz. We used python scripts with the open-source library of larq [ 28 ] and keras [ 29 ]. The TensorFlow backend engine was employed, and the NVIDIA GeForce GTX 1060 3GB was applied for acceleration.…”

Section: Research Methodsmentioning

confidence: 99%

FPGA-Based Acceleration on Additive Manufacturing Defects Inspection

Luo

Chen

2021

Sensors

View full text Add to dashboard Cite

Additive manufacturing (AM) has gained increasing attention over the past years due to its fast prototype, easier modification, and possibility for complex internal texture devices when compared to traditional manufacture processing. However, potential internal defects are occurring during AM processes, and it requires real-time inspections to minimize the costs by either aborting the processing or repairing the defect. In order to perform the defects inspection, first the defects database NEU-DET is used for training. Then, a convolution neural network (CNN) is applied to perform defects classification. For real-time purposes, Field Programmable Gate Arrays (FPGAs) are utilized for acceleration. A binarized neural network (BNN) is proposed to best fit the FPGA bit operations. Finally, for the image labeled with defects, the selective search and non-maximum algorithms are implemented to help locate the coordinates of defects. Experiments show that the BNN model on NEU-DET can achieve 97.9% accuracy in identifying whether the image is defective or defect-free. As for the image classification speed, the FPGA-based BNN module can process one image within 0.5 s. The BNN design is modularized and can be duplicated in parallel to fully utilize logic gates and memory resources in FPGAs. It is clear that the proposed FPGA-based BNN can perform real-time defects inspection with high accuracy and it can easily scale up to larger FPGA implementations.

show abstract

Larq: An Open-Source Library for Training Binarized Neural Networks

Cited by 74 publications

References 7 publications

VC dimension of partially quantized neural networks in the overparametrized regime

VC dimension of partially quantized neural networks in the overparametrized regime

An SMT-Based Approach for Verifying Binarized Neural Networks

FPGA-Based Acceleration on Additive Manufacturing Defects Inspection

Contact Info

Product

Resources

About