Progressive Learning for Interactive Surveillance Scenes Retrieval

Meessen, Jerome; Desurmont, Xavier; Delaigle, Jean-François; Vleeschouwer, Christophe De; Macq, Benoı̂t

doi:10.1109/cvpr.2007.383517

Cited by 9 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, this scenario has never been studied in the literature. The few existing MIAL methods focus on bag classification [18][19][20] or select groups of instances in a scenario where there is only one query round [21].…”

Section: Introductionmentioning

confidence: 99%

“…This paper focuses on methods that are suitable for MIAL problems. Although several AL methods exist for single instance learning [17], only a handful of methods have been proposed to address MIAL problems [18][19][20][21]. Single instance active learning (SIAL) methods are not suitable for MIL because: 1) in MIL, instances are grouped in sets or bags, and 2) training instances have weak labels.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Bag-Level Aggregation for Multiple-Instance Active Learning in Instance Classification Problems

Carbonneau

Granger²,

Gagnon³

2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

A growing number of applications, e.g. video surveillance and medical image analysis, require training recognition systems from large amounts of weakly annotated data while some targeted interactions with a domain expert are allowed to improve the training process. In such cases, active learning (AL) can reduce labeling costs for training a classifier by querying the expert to provide the labels of most informative instances. This paper focuses on AL methods for instance classification problems in multiple instance learning (MIL), where data is arranged into sets, called bags, that are weakly labeled. Most AL methods focus on single instance learning problems. These methods are not suitable for MIL problems because they cannot account for the bag structure of data. In this paper, new methods for bag-level aggregation of instance informativeness are proposed for multiple instance active learning (MIAL). The aggregated informativeness method identifies the most informative instances based on classifier uncertainty, and queries bags incorporating the most information. The other proposed method, called clusterbased aggregative sampling, clusters data hierarchically in the instance space. The informativeness of instances is assessed by considering bag labels, inferred instance labels, and the proportion of labels that remain to be discovered in clusters. Both proposed methods significantly outperform reference methods in extensive experiments using benchmark data from several application domains. Results indicate that using an appropriate strategy to address MIAL problems yields a significant reduction in the number of queries needed to achieve the same level of performance as single instance AL methods.Recent years have witnessed substantial advances of machine learning techniques that promise to address many complex large-scale problems that were previously thought intractable. However, in many applications, annotating enough representative training data to train a recognition system is costly, and in such cases, one can resort to AL to reduce the annotation burden [1,2]. Moreover, several applications allow to leverage some targeted interactions with human experts, as needed, to label informative data and drive the training process. AL has been used in various applications to reduce the cost of annotations, e.g., in medical image segmentation [3], text classification [4,5] and visual object detection [6].Alternatively, the cost of annotations can be reduced through weakly supervised learning. It generalizes many kinds of learning paradigms including semi-supervised learning and MIL in partially observable environments or learning from uncertain labels. With MIL, training instances are grouped in sets (commonly referred to as bags), and a label is only provided for an entire set, but not for each individual instance. MIL has also been shown to efficiently reduce annotation costs in several

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

Bag-Level Aggregation for Multiple-Instance Active Learning in Instance Classification Problems

Carbonneau

Granger²,

Gagnon³

2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Therefore, relevance feedback requires to define which part in a result is relevant (or irrelevant). The technique of Meessen et al 19 has removed the dynamic aspect by extracting keyframes from videos and applying relevance feedback technique on these keyframes. Chen et al 6 has limited accident event to one sole relation of vehicles' trajectories.…”

Section: Related Work In Surveillance Video Indexing and Retrievalmentioning

confidence: 99%

Surveillance Video Indexing and Retrieval Using Object Features and Semantic Events

Le¹,

Thonnat²,

Boucher

et al. 2009

Int. J. Patt. Recogn. Artif. Intell.

View full text Add to dashboard Cite

In this paper, we propose an approach for surveillance video indexing and retrieval. The objective of this approach is to answer five main challenges we have met in this domain: (1) the lack of means for finding data from the indexed databases, (2) the lack of approaches working at different abstraction levels, (3) imprecise indexing, (4) incomplete indexing, (5) the lack of user-centered search. We propose a new data model containing two main types of extracted video contents: physical objects and events. Based on this data model, we present a new rich and flexible query language. This language works at different abstraction levels, provides both exact and approximate matching and takes into account users' interest. In order to work with the imprecise indexing, two new methods respectively for object representation and object matching are proposed. Videos from two projects which have been partially indexed are used to validate the proposed approach. We have analyzed both query language usage and retrieval results. The obtained retrieval results analyzed by the average normalized ranks are promising. The retrieval results at the object level are compared with another state of the art approach.

show abstract

“…The multiple-instance learning algorithm utilizes the class label from bags for predicting the class label for unseen bags as well as instances. The incorporation of relevance feedback and multiple-instance learning perfectly matches the surveillance video retrieval scenario [10,11] since the former reduces the semantic gap by incorporating the user's high-level perception and gathering training samples in a progressive manner, while the latter guesses the event of interest by analyzing the collected training samples based on incomplete training label information. It should be pointed out that the proposed framework is different from [11] in that our framework not only incorporates RF and MIL into the retrieval, but also tightly integrates them with CHMM and takes advantage of CHMM to model various kinds of human interactions.…”

Section: Introductionmentioning

confidence: 95%

“…al [2] proposes a framework that learns the incident models by clustering trajectories hierarchically with the use of spatial and temporal information [2]. Very few other frameworks borrow the concept of relevance feedback (RF) from the content-based image retrieval (CBIR) to learn the incident models in a progressive fashion, including our previous work [10] and the work proposed by Meessen et al [11]. As a supervised learning technique, relevance feedback incorporates the subjective perceptions from users with the learning process, which significantly increases the retrieval accuracy.…”

Section: Introductionmentioning

confidence: 99%

A Multiple Instance Learning and Relevance Feedback Framework for Retrieving Abnormal Incidents in Surveillance Videos

Zhang¹,

Chen²,

Chen³

et al. 2010

JMM

View full text Add to dashboard Cite

This paper incorporates coupled hidden Markov models (CHMM) with relevance feedback (RF) and multiple-instance learning (MIL) for retrieving various abnormal events in surveillance videos. CHMM is suitable for modeling not only the object’s behavior itself but also the interactions between objects. In addition, to address the challenges posed by the “semantic gap” between high level human concepts and the machine-readable low level visual features, we introduce relevance feedback (RF) to bridge the semantic gap by progressively collecting feedback from the user, which allows the machine to discover the semantic meanings of an event by exploring the patterns behind lowlevel features. The adopted multiple-instance learning algorithm enables the proposed framework to provide a user-friendly video retrieval platform with the use of queryby- example (QBE) interface. The experimental results show the effectiveness of the proposed framework in detecting “chasing”, “fighting”, and “robbery” events by demonstrating the increase of retrieval accuracy through iterations and comparing with other methods. By tightly integrating these key components in a learning system, we ease the surveillance video retrieval problem.

show abstract

Progressive Learning for Interactive Surveillance Scenes Retrieval

Cited by 9 publications

References 19 publications

Bag-Level Aggregation for Multiple-Instance Active Learning in Instance Classification Problems

Bag-Level Aggregation for Multiple-Instance Active Learning in Instance Classification Problems

Surveillance Video Indexing and Retrieval Using Object Features and Semantic Events

A Multiple Instance Learning and Relevance Feedback Framework for Retrieving Abnormal Incidents in Surveillance Videos

Contact Info

Product

Resources

About