Automatically labeling video data using multi-class active learning

Yan, Victoria C.; Yang, Jie; Hauptmann,

doi:10.1109/iccv.2003.1238391

Cited by 123 publications

(80 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In summary, the proposed algorithm (in green) outperforms the other algorithms in each metric in 19 out of 24 scenes. 97 …”

Section: Quantitative Results and Comparisons For The Single Moving Omentioning

confidence: 99%

“…Active learning is a well established subfield of machine learning, which has been shown to benefit a number of computer vision applications such as object categorization [48], image retrieval [31,101], video classification [97], dataset annotation [16], and interactive cosegmentation [7]; maximizing the knowledge gain while valuing the user effort [93]. However, such an algorithm has not been proposed for image based modeling.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Putting the User in the Loop for Image-Based Modeling

et al. 2014

View full text Add to dashboard Cite

Image-based modeling, the task of recovering the 3D structure of an object or a scene using 2D images is one of the primary goals of computer vision. However, 3D reconstruction in practice is a hard task due to the numerous scene irregularities. When a scene is captured from multiple viewpoints, irregularities such as textureless surfaces, specularities, thin structures, etc., make the reconstruction inaccurate. In addition, image sequences or videos of a scene captured in the wild often consist of dynamic or moving objects such as people, which makes the task of image-based modeling extremely hard. In this thesis, our goal is to tackle these problems to obtain a more accurate reconstruction of the scene. In particular, using the intuition that humans easily discern the 3D structure behind the photons, this thesis addresses these problems by putting the user in the loop with the image-based-modeling algorithm.In the first part of the thesis, we focus on image-based modeling of static scenes. We explore putting the user in the loop interacting with the algorithm by providing constraints via simple interactions on the image data, to help overcome the ill-effects of the scene irregularities. We introduce two algorithms within this framework. First, an algorithm where the user drives the process of image-based modeling. Second, a novel active-learning algorithm, which initiates the process of image-based modeling via an unsupervised algorithm and then guides the user to provide constraints when and where needed.In the second part of the thesis, we focus on image-based modeling of dynamic scenes captured by either a dynamic camera or a static camera. We develop algorithms that leverage the dynamic objects in the scene to aid the imagebased modeling algorithm. We propose novel algorithms that use the sparse, yet strong occlusion cues between the moving object and the scene, along with a realistic motion prior for the moving object to recover the depth of the scene.We explore putting the user in the loop within the framework, guided by the algorithm to provide useful depth-order constraints and show that we further improve the solution of the unsupervised algorithm.Finally, we present an end user application: iModel, which puts the user in the loop for object of interest 3D modeling on a mobile device and 3D printing of an object of interest. THESIS COMMITTEE

show abstract

“…In summary, the proposed algorithm (in green) outperforms the other algorithms in each metric in 19 out of 24 scenes. 97 …”

Section: Quantitative Results and Comparisons For The Single Moving Omentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Putting the User in the Loop for Image-Based Modeling

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Previous work on multiclass active learning [5] advocates a querying function closely related to this definition of multiclass margin whereŷ = argmax y ∈Y f y (x) represents the predicted label andỹ = argmax y ∈Y\ŷ f y (x) represents the label corresponding to the second highest activation value,…”

Section: Margin-based Active Learningmentioning

confidence: 99%

“…We first describe a general framework for modeling structured output classifiers, following the approach of incorporating output variable interdependencies directly into a discriminative learning model [2,3]. We then proceed by describing previous margin-based active learning approaches based on the output of linear classifiers [4,5].…”

Section: Preliminariesmentioning

confidence: 99%

See 1 more Smart Citation

Margin-Based Active Learning for Structured Output Spaces

Roth

Small

2006

Lecture Notes in Computer Science

108

View full text Add to dashboard Cite

Abstract. In many complex machine learning applications there is a need to learn multiple interdependent output variables, where knowledge of these interdependencies can be exploited to improve the global performance. Typically, these structured output scenarios are also characterized by a high cost associated with obtaining supervised training data, motivating the study of active learning for these situations. Starting with active learning approaches for multiclass classification, we first design querying functions for selecting entire structured instances, exploring the tradeoff between selecting instances based on a global margin or a combination of the margin of local classifiers. We then look at the setting where subcomponents of the structured instance can be queried independently and examine the benefit of incorporating structural information in such scenarios. Empirical results on both synthetic data and the semantic role labeling task demonstrate a significant reduction in the need for supervised training data when using the proposed methods.

show abstract