A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection

Jiang, Mingjun; Shimasaki, Kohei; Hu, Shaopeng; Senoo, Taku; Ishii, Idaku

doi:10.1109/lra.2020.3048653

Cited by 14 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…High-speed vision is particularly crucial for achieving high-frequency visual feedback, further essential for high-magnification tracking where high-speed pan-tilt adjustment is needed. For example, Jiang et al introduced a 500 fps frame-by-frame object tracking system combining CNN-based object detection and KCF-based tracking [15]. However, the template of the KCF tracker often struggles to handle situations with rapid changes in object size.…”

Section: Development Of Object Tracking Algorithmsmentioning

confidence: 99%

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep

Zhang,

Shimasaki,

Ishii

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

Active vision systems (AVSs) have been widely used to obtain high-resolution images of objects of interest. However, tracking small objects in high-magnification scenes is challenging due to shallow depth of field (DoF) and narrow field of view (FoV). To address this, we introduce a novel high-speed AVS with a continuous autofocus (C-AF) approach based on dynamic-range focal sweep and a high-frame-rate (HFR) frame-by-frame tracking pipeline. Our AVS leverages an ultra-fast pan-tilt mechanism based on a Galvano mirror, enabling high-frequency view direction adjustment. Specifically, the proposed C-AF approach uses a 500 fps high-speed camera and a focus-tunable liquid lens operating at a sine wave, providing a 50 Hz focal sweep around the object’s optimal focus. During each focal sweep, 10 images with varying focuses are captured, and the one with the highest focus value is selected, resulting in a stable output of well-focused images at 50 fps. Simultaneously, the object’s depth is measured using the depth-from-focus (DFF) technique, allowing dynamic adjustment of the focal sweep range. Importantly, because the remaining images are only slightly less focused, all 500 fps images can be utilized for object tracking. The proposed tracking pipeline combines deep-learning-based object detection, K-means color clustering, and HFR tracking based on color filtering, achieving 500 fps frame-by-frame tracking. Experimental results demonstrate the effectiveness of the proposed C-AF approach and the advanced capabilities of the high-speed AVS for magnified object tracking.

show abstract

Section: Development Of Object Tracking Algorithmsmentioning

confidence: 99%

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep

Zhang,

Shimasaki,

Ishii

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed system provides a unique capability for capturing both wide-field and detailed views of a scene simultaneously. To enable the tracking of specific objects in complex backgrounds, a hybrid tracking method that combines convolutional neural networks (CNN) and traditional object tracking techniques was proposed in [21]. This hybrid method achieves high-speed tracking performance of up to 500 fps and has shown promising results in various applications, such as robotics and surveillance.…”

Section: High-speed Visionmentioning

confidence: 99%

“…A reflective PTZ camera system switches perspectives to track multiple objects at 500 fps per second by implementing 2-ms-latency visual feedback control. The high-speed vision feedback control relies on CNN-based hybrid detection methods [21]. Compared with the previous system, this work achieves the following: (1) the acquisition of images with large field of view and high resolution, (2) simultaneous observation of up to 20 objects at a speed of 25 fps; and (3) active tracking of multiple fast-moving objects with no-latency detection.…”

Section: Introductionmentioning

confidence: 99%

An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection

Shimasaki

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

This study proposes a visual tracking system that can detect and track multiple fast-moving appearance-varying targets simultaneously with 500 fps image processing. The system comprises a high-speed camera and a pan-tilt galvanometer system, which can rapidly generate large-scale high-definition images of the wide monitored area. We developed a CNN-based hybrid tracking algorithm that can robustly track multiple high-speed moving objects simultaneously. Experimental results demonstrate that our system can track up to three moving objects with velocities lower than 30 m per second simultaneously within an 8-m range. The effectiveness of our system was demonstrated through several experiments conducted on simultaneous zoom shooting of multiple moving objects (persons and bottles) in a natural outdoor scene. Moreover, our system demonstrates high robustness to target loss and crossing situations.

show abstract

“…Tests are carried out to determine the system performance of the Pan-Tilt system that has been designed [18][19] [20][21] [22][23] [24]. Analysis of the needs for the development of tools will be made as follows: a.…”

Section: B Camera Mechanical Control System Hardware Designmentioning

confidence: 99%

Development of an Optronic Aiming System for Target Tracking on the S60 57mm Cannon Weapon Control System Using a Camera

Andriyan¹,

Kusuma

2023

IJMEIR

View full text Add to dashboard Cite

⎯The development of technological science towards the defense of a country is growing rapidly. The country needs increasingly advanced defense technology but is constrained by an increasingly large budget due to dependence on producing countries. The state provides opportunities for technocrats to carry out research that can later create a defense technology that is inexpensive and does not burden the state budget. The artificial S60 57mm cannon is one of the cannons that functions as an air attack deterrent. The operating system of this weapon is still manual. This research will design and prototype an automatic Weapon Control System on the S60 57mm gun. In this development using the Atmega 8 microcontroller is the controller and interface for transferring data from the camera (Optronic) to the servo motor. The result of the camera position (optronics) will be followed in real-time by the gun barrel. In designing a target tracking control system automatically using the proportional control method. The results of testing the ability of the servo motor as a camera driver to follow targets or moving objects with a maximum angular speed of 15.5 degrees/second at a speed of 0.3 seconds, the average frame rate of the camera are 60 fps. c. Based on testing the servo motor using the Atmega8 microcontroller support, the reading of the angle direction of the servo motor is displayed in the Delphi software. Based on the test results, there is a difference between mathematical calculations and measurements using an arc ruler. With mathematical calculations, there is a difference with a value of 1.06 degrees per 1 degree.

show abstract

A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection

Cited by 14 publications

References 33 publications

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep

High-Magnification Object Tracking with Ultra-Fast View Adjustment and Continuous Autofocus Based on Dynamic-Range Focal Sweep

An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection

Development of an Optronic Aiming System for Target Tracking on the S60 57mm Cannon Weapon Control System Using a Camera

Contact Info

Product

Resources

About