2019 IEEE/CVF International Conference on Computer Vision (ICCV) 2019
DOI: 10.1109/iccv.2019.00066
|View full text |Cite
|
Sign up to set email alerts
|

A Delay Metric for Video Object Detection: What Average Precision Fails to Tell

Abstract: Average precision (AP) is a widely used metric to evaluate detection accuracy of image and video object detectors.In this paper, we analyze object detection from videos and point out that AP alone is not sufficient to capture the temporal nature of video object detection. To tackle this problem, we propose a comprehensive metric, average delay (AD), to measure and compare detection delay. To facilitate delay evaluation, we carefully select a subset of ImageNet VID, which we name as ImageNet VIDT with an emphas… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

0
13
0

Year Published

2020
2020
2022
2022

Publication Types

Select...
5
3
2

Relationship

0
10

Authors

Journals

citations
Cited by 39 publications
(23 citation statements)
references
References 30 publications
0
13
0
Order By: Relevance
“…Besides, densely annotated dataset can benefit for delay evaluation. Recently, average delay(AD) [45] is presented as a new metric to measure the detection delay of video object detectors. We want to know whether AD is suitable for video taken by drones, since the low resolution and wide FoV caused by long distance may hinder the delay measurement.…”
Section: Discussionmentioning
confidence: 99%
“…Besides, densely annotated dataset can benefit for delay evaluation. Recently, average delay(AD) [45] is presented as a new metric to measure the detection delay of video object detectors. We want to know whether AD is suitable for video taken by drones, since the low resolution and wide FoV caused by long distance may hinder the delay measurement.…”
Section: Discussionmentioning
confidence: 99%
“…Some paper proposes the special video metrics. Mao et al propose a novel metric called Average Delay to comprehensively evaluate video object detection [188]. Delay refers to the number of frames from when an object appears to when it is detected.…”
Section: The Evaluation Methods Of Video Detectionmentioning
confidence: 99%
“…It is therefore important that it not only detects with high accuracy, but with low latency, especially when dealing with moving vehicles. Other studies confirm that AP alone is not enough to evaluate object detectors, particularly when it comes to video object detection [39]. Consequently, frames per second (FPS) is another metric used to evaluate the proposed system.…”
Section: P Recision = T P T P +mentioning
confidence: 95%