Subpixel Displacement Measurement at 784 FPS: From Algorithm to Hardware System

Ду, Сонглин; Gu, Kaidong; Ikenaga, Takeshi

doi:10.1109/tim.2022.3162290

Cited by 9 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many previous studies have already explored the use of Field-Programmable Gate Arrays (FPGAs) to develop high frame rate and ultra-low delay systems in various fields. Examples include object tracking [2], straight-line detection [3], and subpixel displacement measurement [4].…”

Section: Introductionmentioning

confidence: 99%

Grid Sample Based Temporal Iteration for Fully Pipelined 1-ms SLIC Superpixel Segmentation System

LI,

HU,

FUCHIKAMI

et al. 2024

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

A 1 millisecond (1-ms) vision system, which processes videos at 1000 frames per second (FPS) within 1 ms/frame delay, plays an increasingly important role in fields such as robotics and factory automation. Superpixel as one of the most extensively employed image oversegmentation methods is a crucial pre-processing step for reducing computations in various computer vision applications. Among the different superpixel methods, simple linear iterative clustering (SLIC) has gained widespread adoption due to its simplicity, effectiveness, and computational efficiency. However, the iterative assignment and update steps in SLIC make it challenging to achieve high processing speed. To address this limitation and develop a SLIC superpixel segmentation system with a 1 ms delay, this paper proposes grid sample based temporal iteration. By leveraging the high frame rate of the input video, the proposed method distributes the iterations into the temporal domain, ensuring that the system's delay keeps within one frame. Additionally, grid sample information is added as initialization information to the obtained superpixel centers for enhancing the stability of superpixels. Furthermore, a selective label propagation based pipeline architecture is proposed for parallel computation of all the possibilities of label propagation. This eliminates data dependency between adjacent pixels and enables a fully pipelined system. The evaluation results demonstrate that the proposed superpixel segmentation system achieves boundary recall and under-segmentation error comparable to the original SLIC algorithm. When considering label consistency, the proposed system surpasses the performance of state-of-the-art superpixel segmentation methods. Moreover, in terms of hardware performance, the proposed system processes 1000 FPS images with 0.985 ms/frame delay.

show abstract

Section: Introductionmentioning

confidence: 99%

Grid Sample Based Temporal Iteration for Fully Pipelined 1-ms SLIC Superpixel Segmentation System

LI,

HU,

FUCHIKAMI

et al. 2024

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The power consumption of FPGAs is significantly lower than that of GPUs. There have been several studies that have used FPGAs in various fields [7][8][9][10]. For FA oriented works, subpixel displacement measurement [8] and straight-line detection [9] have been applied to artificial objects.…”

Section: Introductionmentioning

confidence: 99%

“…There have been several studies that have used FPGAs in various fields [7][8][9][10]. For FA oriented works, subpixel displacement measurement [8] and straight-line detection [9] have been applied to artificial objects. However, few works have focused on natural objects such as tomatoes in this field.…”

Section: Introductionmentioning

confidence: 99%

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Li,

Hu,

Fuchikami

et al. 2024

IET Image Processing

Self Cite

View full text Add to dashboard Cite

A 1 millisecond (1‐ms) vision system that guarantees high efficiency and timely response for tomato defect detection is essential for factory automation. Because of various defect appearances, recently many existing researches focus on CNN based defect detection, but few of them attempt to reach high processing speed to adapt to the factorial assembly line. This paper proposes a global to multi‐scale local based parallel architecture with hardwired CNN for tomato defect detection. This architecture breaks down image‐wise detection into pixel‐wise localization and block‐wise classification. The pixel‐wise localization utilizes tomato‐aware information as constraints for localization performance. The block‐wise classification uses a fully pipelined network structure to obtain the classification result for each block as the pixel stream moves through the network. The classification network has a six‐layer lightweight network structure with quantization for hardwired type implementation on FPGA. The experiment results show that the proposed architecture processes 1000 FPS images with 0.9476 ms/frame delay. And for detection performance, this architecture keeps at 80.18%, only 1.31% lower than ResNet50 based detection system.

show abstract

“…Low latency directly leads to productivity in manufacturing. Many high frame rate and ultra-low vision systems, including straight-line detection [4], dual-hand tracking [5], subpixel displacement [6], realize FPGA implementation. A related work aiming to build a high frame rate (≥ 1000 fps) and ultra-low delay (≤ 1ms/frame) matching system was proposed by Hu and Ikenaga [7].…”

Section: Introductionmentioning

confidence: 99%

Multi-line buffer based pipeline architecture with junction connectivity analysis for high frame rate and ultra-low delay contour-based corner detection

Zha¹,

Li²,

Hu³

et al. 2023

Third International Conference on Computer Vision and Information Technology (CVIT 2022)

View full text Add to dashboard Cite

High frame rate and ultra-low delay corner detection plays an increasingly important role in factory automated scenarios with a demand for accurate and robust corner features. However, classic intensity-based corner detection like the Harris method has limitations in determining corner types and parameter selection. Conventional contour-based corner detection like Chord to Triangular Arms Ratio (CTAR) method uses global level curve extraction based on the whole frame, leading to high delay. Achieving corner detection nearly simultaneous with capturing the same image provides a workable solution to minimize the delay. To modify the conventional detection methods which arbitrarily process any pixels within the scope of the entire input, a multi-line buffer based pipeline architecture is proposed. Using this pipeline, the whole frame is divided into lines processed independently. Junction connectivity analysis is proposed to define corner types based on the architecture. The proposed algorithm almost keeps the robustness (Average Repeatability of 0.5715, Localization Error of 0.4285) with the original CTAR method (AR of 0.5832, LE of 0.4374), better than the Harris method (AR of 0.5322, LE of 0.7324).

show abstract

Subpixel Displacement Measurement at 784 FPS: From Algorithm to Hardware System

Cited by 9 publications

References 39 publications

Grid Sample Based Temporal Iteration for Fully Pipelined 1-ms SLIC Superpixel Segmentation System

Grid Sample Based Temporal Iteration for Fully Pipelined 1-ms SLIC Superpixel Segmentation System

Global to multi‐scale local architecture with hardwired CNN for 1‐ms tomato defect detection

Multi-line buffer based pipeline architecture with junction connectivity analysis for high frame rate and ultra-low delay contour-based corner detection

Contact Info

Product

Resources

About