A New Evaluation Approach for Video Processing Algorithms

Abstract

We present a new evaluation methodology to better evaluate video processing performance. Recent evaluation methods [10], [9], [11] depend heavily on the benchmark dataset. The result may be different if we change the testing video sequences. The difference is mainly due to the video sequence content which usually includes many video processing problems (illumination changes, weak contrast etc.) at different difficulty levels. Hence it is difficult to extrapolate the evaluation result on new sequences.In this paper, we propose an evaluation methodology that help to reuse the evaluation result. We try to isolate each video processing problem and define quantitative measures to compute the difficulty level of a video relatively to the given problem. The maximum difficulty level of the videos at which the algorithm is performing good enough is defined as the upper bound of the algorithm capacity for handling the problem. To illustrate this methodology, we present metrics that evaluate the algorithm performance relatively to the problems of handling weakly contrasted objects and shadows.

ETISEO, performance evaluation for video surveillance systems

“…Moreover, we will add objective metrics to quantify automatically the difficulty levels of video processing problems in videos to facilitate the video characterization and selection. These metrics will enable to generalize the evaluation results for new scenes as described in [10]. After that we will study the interdependency between video processing problems to extend the ETISEO results to a combination of problems.…”

“…Moreover, the prediction of algorithm performance on new scenes based on these evaluation results is difficult because we have to compare these new scenes with the ETISEO video sequences. To solve this problem, we are currently working on defining objective and quantitative metrics to measure automatically the difficulty levels of video processing problems [10].…”

Activity recognition using Video Event Segmentation with Text (VEST)

“…The metrics were compiled for detection, localization, tracking, classification, and activity/event recognition. 16 VEST applies and extends the video analysis from the PETS and other challenges to deal with the peculiarities of organizing multi-modal information collected over video and text information.…”

Video Event Detection and Human Behavior Analysis

“…then event detection and behavioral analysis are done. Central processor can be integrated into third-party video surveillance platform, it is linked with mainstream video surveillance platform seamlessly [11,12]. Event analysis results and related parameter settings also communicate through intelligent video transmission protocol and with the video surveillance platform.…”