A High-Throughput and Power-Efficient FPGA Implementation of YOLO CNN for Object Detection

Nguyen, Duy Thanh; Nguyen, Tuan Nghia; Kim, Hyun; Lee, Hyuk-Jae

doi:10.1109/tvlsi.2019.2905242

Cited by 334 publications

(144 citation statements)

References 19 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…clustering method to obtain anchor boxes by suppressing the interference boxes [26]. More speci cally, the K-means algorithm randomly selects bounding boxes as the initial clustering center, and the distance between each bounding box to the cluster center is calculated.…”

Section: Bounding Box Generationmentioning

confidence: 99%

Video-Based Detection Infrastructure Enhancement for Automated Ship Recognition and Behavior Analysis

Chen

Yang

et al. 2020

Journal of Advanced Transportation

View full text Add to dashboard Cite

Video-based detection infrastructure is crucial for promoting connected and autonomous shipping (CAS) development, which provides critical on-site traffic data for maritime participants. Ship behavior analysis, one of the fundamental tasks for fulfilling smart video-based detection infrastructure, has become an active topic in the CAS community. Previous studies focused on ship behavior analysis by exploring spatial-temporal information from automatic identification system (AIS) data, and less attention was paid to maritime surveillance videos. To bridge the gap, we proposed an ensemble you only look once (YOLO) framework for ship behavior analysis. First, we employed the convolutional neural network in the YOLO model to extract multi-scaled ship features from the input ship images. Second, the proposed framework generated many bounding boxes (i.e., potential ship positions) based on the object confidence level. Third, we suppressed the background bounding box interferences, and determined ship detection results with intersection over union (IOU) criterion, and thus obtained ship positions in each ship image. Fourth, we analyzed spatial-temporal ship behavior in consecutive maritime images based on kinematic ship information. The experimental results have shown that ships are accurately detected (i.e., both of the average recall and precision rate were higher than 90%) and the historical ship behaviors are successfully recognized. The proposed framework can be adaptively deployed in the connected and autonomous vehicle detection system in the automated terminal for the purpose of exploring the coupled interactions between traffic flow variation and heterogeneous detection infrastructures, and thus enhance terminal traffic network capacity and safety.

show abstract

Section: Bounding Box Generationmentioning

confidence: 99%

Video-Based Detection Infrastructure Enhancement for Automated Ship Recognition and Behavior Analysis

Chen

Yang

et al. 2020

Journal of Advanced Transportation

View full text Add to dashboard Cite

show abstract

“…is the maximum size of shortcut's output in the second group. As shown in Table I in [9], for layers in [1,i], row_buffl = (Kl+1) × Nl × Hl × QA, out_buffl = To × × QS, and for layers in [i+1,L], in_frame_buffl = Hl 2 × Nl × QA, out_buffl = To × Hl 2 × QS, respectively. , , , are the kernel size, feature map width (height), number of input channels, and number of output channels, respectively.…”

Section: B Optimization For the Mixed Data Flow Designmentioning

confidence: 99%

“…It is assumed that the provided DRAM bandwidth is sufficient to not affect the execution time. In fact, this assumption is true in most design Table I in [9]. It should be noted that Scheme 1 (no reuse) in [9] is not listed here.…”

Section: =1mentioning

confidence: 99%

See 2 more Smart Citations

Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors

Nguyen

Kim

Lee

2021

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) require both intensive computation and frequent memory access, which lead to a low processing speed and large power dissipation. Although the characteristics of the different layers in a CNN are frequently quite different, previous hardware designs have employed common optimization schemes for them. This paper proposes a layer-specific design that employs different organizations that are optimized for the different layers. The proposed design employs two layer-specific optimizations: layer-specific mixed data flow and layer-specific mixed precision. The mixed data flow aims to minimize the off-chip access while demanding a minimal on-chip memory (BRAM) resource of an FPGA device. The mixed precision quantization is to achieve both a lossless accuracy and an aggressive model compression, thereby further reducing the offchip access. A Bayesian optimization approach is used to select the best sparsity for each layer, achieving the best trade-off between the accuracy and compression. This mixing scheme allows the entire network model to be stored in BRAMs of the FPGA to aggressively reduce the off-chip access, and thereby achieves a significant performance enhancement. The model size is reduced by 22.66-28.93 times compared to that in a full-precision network with a negligible degradation of accuracy on VOC, COCO, and ImageNet datasets. Furthermore, the combination of mixed dataflow and mixed precision significantly outperforms the previous works in terms of both throughput, off-chip access, and on-chip memory requirement.

show abstract

EPA: The effective pipeline architecture for CNN accelerator with high performance and computing efficiency based on FPGA

Zhang

Yin

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Thanks to the great developments of the latest Field Programmable Gate Array (FPGA), the performance bottleneck of Deep Learning hardware accelerators has been converted to computing ability. In this paper, a novel FPGA‐based Convolutional Neural Network (CNN) Accelerator architecture, named the Effective Pipeline Architecture (EPA) is proposed to optimize the resource usage for the implementation of the CNN calculation. As the unique storage strategies, which contain many creative designing details, are adopted and optimized for different CNN models and layers, great DSP computing efficiency can be achieved in the fine‐grained pipeline. Moreover, compared with the traditional architectures, through the kernel combination and data scheduling, twice throughput for the general matrix multiplication is realized in a great many parallel DSP48E resources. As a result, the realization of Yolov2‐Tiny achieves 873 Giga Operations Per Second (GOPS) by 902 DSPs with 67 Frames Per Second (FPS), and the computing efficiency in most layers can even reach more than 90%, which improves the calculation performance and efficiency comparing with the previous designs, and is significant to meet the increasing computing requirement.

show abstract

A High-Throughput and Power-Efficient FPGA Implementation of YOLO CNN for Object Detection

Cited by 334 publications

References 19 publications

Video-Based Detection Infrastructure Enhancement for Automated Ship Recognition and Behavior Analysis

Video-Based Detection Infrastructure Enhancement for Automated Ship Recognition and Behavior Analysis

Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors

EPA: The effective pipeline architecture for CNN accelerator with high performance and computing efficiency based on FPGA

Contact Info

Product

Resources

About