Efficient Object Localization and Segmentation in Weakly Labeled Videos

Abstract

Abstract. In this paper, we tackle the problem of efficiently segmenting objects in weakly labeled videos. Internet videos (e.g., YouTube) are often associated with a semantic tag describing the main object within the video. However, this tag does not provide any spatial or temporal information about the object within the video. So these videos are weakly labeled. We propose a novel and efficient approach to localize the object of interest within the video and perform pixel-level segmentation. Given a video with an object tag, our proposed method automatically localizes the object and segments it from the background in each frame of the video. Our method combines object appearance modeling and temporal consistency among frames in a principled framework. Our method does not require user inputs or object detectors, so it can be potentially applied to videos of any object categories. We evaluate our method on a dataset consisting of more than 100 video shots of 10 different object categories. Our experimental results show that our method outperforms other baseline approaches.

Unsupervised Learning of Foreground Object Segmentation

“…The second main approach to unsupervised learning includes methods for image co-segmentation (Joulin et al (2010); Kim et al (2011); Rubinstein et al (2013); Joulin et al (2012); Kuettel et al (2012); Vicente et al (2011); Rubio et al (2012); Leordeanu et al (2012)) and weakly supervised localization (Deselaers et al (2012);Nguyen et al (2009); Siva et al (2013)). Earlier methods are based on local feature matching and detection of their co-occurrence patterns (Stretcu and Leordeanu (2015); Sivic et al (2005); Leordeanu et al (2005); Parikh and Chen (2007); Liu and Chen ( 2007)), while more recent ones ; Rochan and Wang (2014)) discover object tubes by linking candidate bounding boxes between frames with or without refining their location. Traditionally, the task of unsupervised learning from image sequences has been formulated as a feature matching or data clustering optimization problem, which is computationally very expensive due to its combinatorial nature.…”

Unsupervised Learning from Videos for Object Discovery in Single Images

“…The task of unsupervised object discovery in videos is strongly related to co-localization [11][12][13][14][15] and co-segmentation [16][17][18][19][20][21][22][23]. The task has been studied for more than a decade in computer vision, with initial works mainly being based on local feature matching and detection of their co-occurring patterns [24][25][26][27].…”

“…The task has been studied for more than a decade in computer vision, with initial works mainly being based on local feature matching and detection of their co-occurring patterns [24][25][26][27]. Recent approaches [12,15,18] discovered object tubes by linking candidate detection between frames with or without refining their location. Typically, the task of unsupervised learning from image sequences is formulated as an optimization problem for either feature matching, conditional random fields, or data clustering.…”

Unsupervised Learning from Video to Detect Foreground Objects in Single Images

“…The task of object discovery and unsupervised learning in video is related to co-segmentation [13,18,35,14,20,42,36] and weakly supervised localization [9,25,38]. Earlier methods are based on local feature matching and detection of their co-occurrences patterns [40,39,21,27,23], while recent approaches [15,32] discover object tubes by linking candidate detections between frames with or without refining their location. Traditionally, the task of unsupervised learning from image sequences, formulated as an optimization problem for either feature matching, conditional random fields or data clustering is inherently expensive due to the combinatorial nature of the problem.…”