Hierarchy of Localized Random Forests for Video Annotation

Nagaraja, Naveen Shankar; Ochs, Peter; Liu, Kun; Brox, Thomas

doi:10.1007/978-3-642-32717-9_3

Cited by 9 publications

(6 citation statements)

References 22 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[111] proposes a joint propagation strategy with synthesized training samples. To better account for errors in label propagation, [64] proposes a hierarchy of local classifiers for this task and [5] leverages a mixture-of-tree model for temporal association. The work of [15] leverages label propagation as a data augmentation scheme and demonstrate improved performance on semantic segmentation.…”

Section: Methodsmentioning

confidence: 99%

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Liao¹,

Xie²,

Geiger³

2021

Preprint

View full text Add to dashboard Cite

For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other. Recently, however, the community has realized that progress towards robust intelligent systems such as self-driving cars requires a concerted effort across the different fields. This motivated us to develop KITTI-360, successor of the popular KITTI dataset. KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization to facilitate research at the intersection of vision, graphics and robotics. For efficient annotation, we created a tool to label 3D scenes with bounding primitives and developed a model that transfers this information into the 2D image domain, resulting in over 150k semantic and instance annotated images and 1B annotated 3D points. Moreover, we established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset. KITTI-360 will enable progress at the intersection of these research areas and thus contributing towards solving one of our grand challenges: the development of fully autonomous self-driving systems.

show abstract

Section: Methodsmentioning

confidence: 99%

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Liao¹,

Xie²,

Geiger³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In (14), we show that the weight of each template is determined by the product of their reliability and discriminative strength.…”

Section: )Udph 1xpehumentioning

confidence: 99%

“…While some algorithms combine both color and shape cues [4], [5] for video segmentation, existing methods do not take the reliability of shape 1: judge the motion scale, and generate the target and template regions 2: determine the template grid scale via (12) 3: solve σ by substituting (13) into (14), then obtain p u obj (x) and p u bck (x) with the computed σ 4: compute the reliability of the template region in the u-th template Cu(x) via (16) 5: obtain the color probability Pc(x|F ) and total discriminative strength D(x) for pixel x via (16) and (17) Output: color probability Pc(x|F ) and discriminative strength D(x) model into consideration, which makes the integration results less accurate. In this work, we exploit the reliability of both color and shape models effectively to generate the shape and color prior for segmentation.…”

Section: Integrating Shape and Color Modelsmentioning

confidence: 99%

Interactive Video Segmentation via Local Appearance Model

Sun

2017

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

In numerous video segmentation algorithms, shape and color priors from previous frames are propagated to successive frames for processing. One prime issue of the existing algorithms is how priors are modeled and propagated effectively. This paper proposes a novel algorithm for accurate and robust foreground prediction via a local appearance model based on shape and color. In the shape estimation process, instead of performing global matching, a local search mechanism is developed to capture complex motions. In addition, global information is used to facilitate local matching for the final shape estimation. Color cues from multiple frames are used to estimate the foreground (background) pixel distribution in the current image. Furthermore, the contribution of each frame is weighted based on its reliability and discriminative strength. Based on the proposed local appearance model, a graph cut algorithm is used to generate segmentation results. Experimental results show that accurate video segmentation can be obtained by the proposed algorithm with fewer user interactions.

show abstract

“…These methods either assume that we have annotations on a few frames [5,26] or leave the ambiguous areas to human labeling [9]. Galasso [13] applies an off-line trained model to label all the region trajectories belonging to pedestrians.…”

Section: Related Workmentioning

confidence: 99%

“…This technique has a broad range of applications such as surveillance [4], video compression G. Zhang ( ) · Z. Yuan · Y. Liu · L. Ma · N. Zheng Institute of Artificial Intelligence and Robotics, #28 West Xianning Road, Xi'an, Shaanxi, People's Republic of China e-mail: zhangtetsu@gmail.com [8], video retrieval [25], and action recognition [15]. According to the applications, video segmentation can be described as followings: removing background from a video clip [33], tracking and extracting specific objects [23], propagating sparse annotations [5,26], and over-segmenting a video into spatiotemporal consistent super-voxels [38,39]. However, compared with image segmentation, video segmentation has more challenges and it is far less sophisticated for industrial needs.…”

Section: Introductionmentioning

confidence: 98%

Video object segmentation by integrating trajectories from points and regions

Zhang¹,

Yuan²,

Liu³

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

We describe a novel video object segmentation system based on a conditional random field model with high-order term which is capable of capturing longer-range spatial and temporal grouping information. Our system is able to segment different moving objects effectively from complex background due to integrating the complementary properties of trajectories from points and regions. Although point and region trajectories have already been used in video object segmentation, their complementary properties have not been well investigated. In this paper, we propose an ingenious scheme to transfer the labels of sparse point trajectories to region trajectories. Especially, for region trajectories with few texture, this scheme can automatically predict their label probabilities by using a Gaussian mixture model of appearance and motion given the labels of point trajectories. Meanwhile, we design a reliability measurement for region trajectories based on shape consistency, which helps us to design robust high-order potentials for spatially overlapping region trajectories. Our region trajectories are extracted from hierarchical image over-segmentation, and hence they can capture meaningful regions over time. Additionally, our approach is a streaming process, in which object labels are propagated over a video. We validate the effectiveness of our approach on public challenging datasets, and show that our approach outperforms other competing methods

show abstract

Hierarchy of Localized Random Forests for Video Annotation

Cited by 9 publications

References 22 publications

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Interactive Video Segmentation via Local Appearance Model

Video object segmentation by integrating trajectories from points and regions

Contact Info

Product

Resources

About