BinaryEye: A 20 kfps Streaming Camera System on FPGA with Real-Time On-Device Image Recognition Using Binary Neural Networks

Jokic, Petar; Emery, Stephane; Benini, Luca

doi:10.1109/sies.2018.8442108

Cited by 19 publications

(25 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Topology [52] 95.7 FC200-3FC100-FC10 [1] 96.0 MLP [51] 97 NK [53] 97.0 LeNet [54] 97.69 MLP [55] 97.86 ConvPool-2 [35] 98. 25 1/4 MLP [41] 98.4 MLP [56] 98.40 MLP [57] 98.6 NK [58] 98.67 MLP [59] 98.77 FC784-3FC512-FC10 Table 6. Accuracies on the MNIST dataset of non-binary networks related to works reviewed.…”

Section: Source Accuracy (%)mentioning

confidence: 99%

See 1 more Smart Citation

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

In this work, we review Binarized Neural Networks (BNNs). BNNs are deep neural networks that use binary values for activations and weights, instead of full precision values. With binary values, BNNs can execute computations using bitwise operations, which reduces execution time. Model sizes of BNNs are much smaller than their full precision counterparts. While the accuracy of a BNN model is generally less than full precision models, BNNs have been closing accuracy gap and are becoming more accurate on larger datasets like ImageNet. BNNs are also good candidates for deep learning implementations on FPGAs and ASICs due to their bitwise efficiency. We give a tutorial of the general BNN methodology and review various contributions, implementations and applications of BNNs.

show abstract

Section: Source Accuracy (%)mentioning

confidence: 99%

“…This type of work flow is called High Level Synthesis (HLS). HLS has been a major component of the research done with BNNs on FPGAs [35,[38][39][40]54,56,63,68,69,72].…”

Section: High Level Synthesismentioning

confidence: 99%

A Review of Binarized Neural Networks

2019

View full text Add to dashboard Cite

show abstract

“…To further apply BNNs to real applications, Jokic et al [20] proposed an FPAG-based 20 kfp streaming camera system called BinaryEye. The system could classify regions of interest within a frame in real-time streaming mode.…”

Section: B Bnn Designsmentioning

confidence: 99%

“…To further analyze the proposed design, an existing BNN architecture [8] used in Jokic et al [20] and Hashimoto et al [21] was also implemented for comparison. In the following sections, we will discuss the BNN configurations, the recognition accuracy and the system performance.…”

Section: System Evaluationmentioning

confidence: 99%

An FPGA-Based Hardware/Software Design Using Binarized Neural Networks for Agricultural Applications: A Case Study

Huang

2021

IEEE Access

View full text Add to dashboard Cite

This work presents an FPGA-based hardware/software design to help the agricultural robot intelligently decide if biological agents need to be applied to the target crops. For target crop recognition, in our global positioning, the selective search integrates with a thresholding scheme to reduce the number of region of interest (ROI) in a captured image. In our local recognition, a binarized neural network (BNN) architecture is presented to help recognize the target crop. Furthermore, an estimation method of pest and disease severity is also presented. Experiments show integrating our presented BNN architecture needs a few extra resources (less than 17% of available FPGA resources in terms of Xilinx Zynq UltraScall+ T M MPSoC ZU3EG A484), compared to an existing BNN one. However, the top-1 accuracy rate and the top-5 one can be increased by 32.25% to 32.84% and by 14.99% to 15.17%, respectively. Furthermore, when the presented BNN architecture was also implemented on the ARM Cortex-A53 CPU and the NVIDIA GeForce RTX 2080 GPU, our BNN hardware module on the FPGA can accelerate the frames per second (FPS) by a factor of 3,690.18 and a factor of 1.07, respectively. INDEX TERMS FPGA, binarized neural networks, object detection, agriculture

show abstract

“…The implementation of CNNs on FPGAs has been studied from the literature to certain extent. More specifically, Binary-Eye in [20] has presented an implementation of binary neural networks on FPGA. The presented implementation can be used in IoT and distributed systems where the stream of images for a camera needs to be processed.…”

Section: Related Workmentioning

confidence: 99%

A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA

Farhadi

Ghasemi

Yang

2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

Nowadays most research in visual recognition using Convolutional Neural Networks (CNNs) follows the "deeper model with deeper confidence" belief to gain a higher recognition accuracy. At the same time, deeper model brings heavier computation. On the other hand, for a large chunk of recognition challenges, a system can classify images correctly using simple models or so-called shallow networks. Moreover, the implementation of CNNs faces with the size, weight, and energy constraints on the embedded devices. In this paper, we implement the adaptive switching between shallow and deep networks to reach the highest throughput on a resource-constrained MPSoC with CPU and FPGA. To this end, we develop and present a novel architecture for the CNNs where a gate makes the decision whether using the deeper model is beneficial or not. Due to resource limitation on FPGA, the idea of partial reconfiguration has been used to accommodate deep CNNs on the FPGA resources. We report experimental results on CIFAR-10, CIFAR-100, and SVHN datasets to validate our approach. Using confidence metric as the decision making factor, only 69.8%, 71.8%, and 43.8% of the computation in the deepest network is done for CIFAR-10, CIFAR-100, and SVHN while it can maintain the desired accuracy with the throughput of around 400 images per second for SVHN dataset.

show abstract

BinaryEye: A 20 kfps Streaming Camera System on FPGA with Real-Time On-Device Image Recognition Using Binary Neural Networks

Cited by 19 publications

References 22 publications

A Review of Binarized Neural Networks

A Review of Binarized Neural Networks

An FPGA-Based Hardware/Software Design Using Binarized Neural Networks for Agricultural Applications: A Case Study

A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA

Contact Info

Product

Resources

About