Video Object Detection Guided by Object Blur Evaluation

Yu-jie, WU; Zhang, Hong; Li, Yawei; Yang, Yifan; Yuan, Di

doi:10.1109/access.2020.3038913

Cited by 24 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Single Frame Baseline [1] Faster RCNN 76.7 ST-Lattice [22] Faster RCNN 79.0 BFAN [70] Faster RCNN 79.1 STCA [71] Faster RCNN 80.3 SELSA [49] Faster RCNN 80.…”

Section: Methodsmentioning

confidence: 99%

TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers

Zhou,

Li,

et al. 2022

Preprint

View full text Add to dashboard Cite

Detection Transformer (DETR) and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors. However, their performance on Video Object Detection (VOD) has not been well explored. In this paper, we present TransVOD, the first end-to-end video object detection system based on spatial-temporal Transformer architectures. The first goal of this paper is to streamline the pipeline of VOD, effectively removing the need for many hand-crafted components for feature aggregation, e.g., optical flow model, relation networks. Besides, benefited from the object query design in DETR, our method does not need complicated post-processing methods such as Seq-NMS. In particular, we present a temporal Transformer to aggregate both the spatial object queries and the feature memories of each frame. Our temporal transformer consists of two components: Temporal Query Encoder (TQE) to fuse object queries, and Temporal Deformable Transformer Decoder (TDTD) to obtain current frame detection results. These designs boost the strong baseline deformable DETR by a significant margin (3 %-4 % mAP) on the ImageNet VID dataset. TransVOD yields comparable performances on the benchmark of ImageNet VID. Then, we present two improved versions of TransVOD including TransVOD++ and TransVOD Lite. The former fuses object-level information into object query via dynamic convolution while the latter models the entire video clips as the output to speed up the inference time. We give detailed analysis of all three models in the experiment part. In particular, our proposed TransVOD++ sets a new state-of-the-art record in terms of accuracy on ImageNet VID with 90.0 % mAP. Our proposed TransVOD Lite also achieves the best speed and accuracy trade-off with 83.7 % mAP while running at around 30 FPS on a single V100 GPU device. Code and models will be available for further research.

show abstract

“…Single Frame Baseline [1] Faster RCNN 76.7 ST-Lattice [22] Faster RCNN 79.0 BFAN [70] Faster RCNN 79.1 STCA [71] Faster RCNN 80.3 SELSA [49] Faster RCNN 80.…”

Section: Methodsmentioning

confidence: 99%

TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers

Zhou,

Li,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Drone-GAN can enhance vehicle features in blurred images, so that vehicles in blurred images can be detected more easily and accurately, which has high robustness towards small target and target occlusion. Compared with our method, Wu et al [52] proposed a video object detection algorithm based on target blurring degree evaluation, which only considers the blurring degree of the target, but a clear target video frame contributes more to the result than a blurred video frame, thus improving the detection performance. However, this algorithm has poor detection performance when the target is highly blurred, small and occluded.…”

Section: Discussionmentioning

confidence: 99%

An Adaptive Deblurring Vehicle Detection Method for High-Speed Moving Drones: Resistance to Shake

Liu

Wang

Qiu

et al. 2021

Entropy

View full text Add to dashboard Cite

Vehicle detection is an essential part of an intelligent traffic system, which is an important research field in drone application. Because unmanned aerial vehicles (UAVs) are rarely configured with stable camera platforms, aerial images are easily blurred. There is a challenge for detectors to accurately locate vehicles in blurred images in the target detection process. To improve the detection performance of blurred images, an end-to-end adaptive vehicle detection algorithm (DCNet) for drones is proposed in this article. First, the clarity evaluation module is used to determine adaptively whether the input image is a blurred image using improved information entropy. An improved GAN called Drone-GAN is proposed to enhance the vehicle features of blurred images. Extensive experiments were performed, the results of which show that the proposed method can detect both blurred and clear images well in poor environments (complex illumination and occlusion). The detector proposed achieves larger gains compared with SOTA detectors. The proposed method can enhance the vehicle feature details in blurred images effectively and improve the detection accuracy of blurred aerial images, which shows good performance with regard to resistance to shake.

show abstract

“…6 for a visual example of each of these techniques for RGB, LWIR and fused RGB-LWIR. The blurred and blurred + ipped IP techniques are especially useful because of video vibrations caused by the oscillatory motions from the airframe's propellers 41 . Model training on blurred images helps to ensure that the model will continue to work when frames are blurred due to camera movement, target object movement, or both.…”

Section: B Data Collectionmentioning

confidence: 99%

Assessing thermal imagery integration into object detection methods on air-based collection platforms

Gallagher

Oughton

2023

Preprint

View full text Add to dashboard Cite

Object detection models commonly focus on utilizing the visible spectrum via Red-Green-Blue (RGB) imagery. Due to various limitations with this approach in low visibility settings, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) (7.5–13.5 µm) images to increase object detection performance. However, we still lack baseline performance metrics evaluating RGB, LWIR and RGB-LWIR fused object detection machine learning models, especially from air-based platforms. This study undertakes such an evaluation finding that a blended RGB-LWIR model generally exhibits superior performance compared to traditional RGB or LWIR approaches. For example, an RGB-LWIR blend only performed 1–5% behind the RGB approach in predictive power across various altitudes and periods of clear visibility. Yet, RGB fusion with a thermal signature overlayed provides edge redundancy and edge emphasis, both which are vital in supporting edge detection machine learning algorithms. This approach has the ability to improve object detection performance for a range of use cases in industrial, consumer, government, and military applications. Finally, this research additionally contributes a novel open labeled training dataset of 6,300 images for RGB, LWIR, and RGB-LWIR fused imagery, collected from air-based platforms, enabling further multispectral machine-driven object detection research.

show abstract

Video Object Detection Guided by Object Blur Evaluation

Cited by 24 publications

References 41 publications

TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers

TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers

An Adaptive Deblurring Vehicle Detection Method for High-Speed Moving Drones: Resistance to Shake

Assessing thermal imagery integration into object detection methods on air-based collection platforms

Contact Info

Product

Resources

About