Highly-Parallel Hardwired Deep Convolutional Neural Network for 1-ms Dual-Hand Tracking

Zhang, Peiqi; Hu, Tingting; Luo, Dingli; Ду, Сонглин; Ikenaga, Takeshi

doi:10.1109/tcsvt.2021.3103784

Cited by 7 publications

(6 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To realize a 1 ms vision system, the hardwired type implementation was proposed [10]. The proposed architecture is a huge, deep pipeline in which all intermediate neural values are represented in hardware states.…”

Section: Cnn Fpga Implementationmentioning

confidence: 99%

“…The power consumption of FPGAs is significantly lower than that of GPUs. There have been several studies that have used FPGAs in various fields [7][8][9][10]. For FA oriented works, subpixel displacement measurement [8] and straight-line detection [9] have been applied to artificial objects.…”

Section: Introductionmentioning

confidence: 99%

“…Optimizing the pipelined streaming strategy improves throughput, but it is not sufficient for high frame rate and ultra-low-delay systems. The architecture of hardwired type, as shown in Figure 2b, was proposed by Zhang et al in their work on highly parallel neural networks on FPGA [10]. This design allowed for a high degree of parallelism by mapping the entire network directly onto an FPGA.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Li,

Hu,

Fuchikami

et al. 2024

IET Image Processing

Self Cite

View full text Add to dashboard Cite

A 1 millisecond (1‐ms) vision system that guarantees high efficiency and timely response for tomato defect detection is essential for factory automation. Because of various defect appearances, recently many existing researches focus on CNN based defect detection, but few of them attempt to reach high processing speed to adapt to the factorial assembly line. This paper proposes a global to multi‐scale local based parallel architecture with hardwired CNN for tomato defect detection. This architecture breaks down image‐wise detection into pixel‐wise localization and block‐wise classification. The pixel‐wise localization utilizes tomato‐aware information as constraints for localization performance. The block‐wise classification uses a fully pipelined network structure to obtain the classification result for each block as the pixel stream moves through the network. The classification network has a six‐layer lightweight network structure with quantization for hardwired type implementation on FPGA. The experiment results show that the proposed architecture processes 1000 FPS images with 0.9476 ms/frame delay. And for detection performance, this architecture keeps at 80.18%, only 1.31% lower than ResNet50 based detection system.

show abstract

Section: Cnn Fpga Implementationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Li,

Hu,

Fuchikami

et al. 2024

IET Image Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Low latency directly leads to productivity in manufacturing. Many high frame rate and ultra-low vision systems, including straight-line detection [4], dual-hand tracking [5], subpixel displacement [6], realize FPGA implementation. A related work aiming to build a high frame rate (≥ 1000 fps) and ultra-low delay (≤ 1ms/frame) matching system was proposed by Hu and Ikenaga [7].…”

Section: Introductionmentioning

confidence: 99%

Multi-line buffer based pipeline architecture with junction connectivity analysis for high frame rate and ultra-low delay contour-based corner detection

Zha¹,

Li²,

Hu³

et al. 2023

Third International Conference on Computer Vision and Information Technology (CVIT 2022)

View full text Add to dashboard Cite

High frame rate and ultra-low delay corner detection plays an increasingly important role in factory automated scenarios with a demand for accurate and robust corner features. However, classic intensity-based corner detection like the Harris method has limitations in determining corner types and parameter selection. Conventional contour-based corner detection like Chord to Triangular Arms Ratio (CTAR) method uses global level curve extraction based on the whole frame, leading to high delay. Achieving corner detection nearly simultaneous with capturing the same image provides a workable solution to minimize the delay. To modify the conventional detection methods which arbitrarily process any pixels within the scope of the entire input, a multi-line buffer based pipeline architecture is proposed. Using this pipeline, the whole frame is divided into lines processed independently. Junction connectivity analysis is proposed to define corner types based on the architecture. The proposed algorithm almost keeps the robustness (Average Repeatability of 0.5715, Localization Error of 0.4285) with the original CTAR method (AR of 0.5832, LE of 0.4374), better than the Harris method (AR of 0.5322, LE of 0.7324).

show abstract

“…In recent decades, machine vision, as a well-known and contactless inspection technology, has been deeply researched and widely applied in numerous fields. With the development of industrial automation, machine vision is becoming one of the key technologies for the next progression of factorial automation, such as sign detection 1 , object tracking 2 , object detection and high precise location 3 . Shape information, as a common geometric information, plays a crucial role in many industrial scenarios.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Relative vectors clustering and temporal constraint based generalized Hough transform for high frame rate and ultra-low delay arbitrary shape detection

Yang

et al. 2023

Third International Conference on Computer Vision and Information Technology (CVIT 2022)

View full text Add to dashboard Cite

Arbitrary shape detection, compared with analytic shape detection, plays a more significant role in machine vision for industrial automation. With the development of industrial automation, the requirements for low-delay detection and high-precision operation are gradually increasing. However, existing works on arbitrary shape detection pay more attention to detection accuracy, but few researchers attempt to achieve ultra-low detection delay, because of the limitation of the bandwidth between memory to CPU. This paper proposes clustering relative-vectors-based parallelization and temporal constraint for generalized Hough transform (GHT) algorithm compression to achieve the ultra-low delay process system, implemented on FPGA. By clustering relative vectors among closed edge pixels as a clustered vector, and defining a regularized R-Table structure, the parallelization of GHT has been increased. Moreover, fully utilizing the temporal information in high frame rate video leads to the compression of accumulator memory consumption, by confining the search widow and restricting the rotation range according to the detection result from the previous frame. The evaluation shows that the proposed architecture finishes the detection in VGA sized sequence with an ultra-low process delay of 1.851ms per frame.

show abstract

Highly-Parallel Hardwired Deep Convolutional Neural Network for 1-ms Dual-Hand Tracking

Cited by 7 publications

References 44 publications

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Multi-line buffer based pipeline architecture with junction connectivity analysis for high frame rate and ultra-low delay contour-based corner detection

Relative vectors clustering and temporal constraint based generalized Hough transform for high frame rate and ultra-low delay arbitrary shape detection

Contact Info

Product

Resources

About