Feature Space Optimization for Semantic Video Segmentation

Kundu, Anjan; Vineet, Vibhav; Koltun, Vladlen

doi:10.1109/cvpr.2016.345

Cited by 169 publications

(143 citation statements)

References 22 publications

Supporting

Mentioning

143

Contrasting

Order By: Relevance

“…filters in Eqn. (14) can be automatically learned from data, if the patterns in a specific region are homogeneous, such as face or human images, which have more regular shapes than images in VOC12.…”

Section: Modeling Smoothness Termsmentioning

confidence: 99%

“…As a result, Eqn. (14) suggests that the probability of object v presented at position j is updated by weighted averaging over the probabilities at its nearby positions. Thus, as shown in Fig.…”

Section: Modeling Smoothness Termsmentioning

confidence: 99%

“…It is defined as a multi-label classification problem, aiming to assign each pixel with a category label. There are two widely adopted research realms, including semantic image segmentation [1], [2], [3], [4], [5], [6], [7] and semantic video segmentation [8], [9], [10], [11], [12], [13], [14]. The former employs a static image as input, while the latter employs a video sequence.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning Markov Random Field for Semantic Segmentation

Liu

Luo

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

143

107

View full text Add to dashboard Cite

Abstract-Semantic segmentation tasks can be well modeled by Markov Random Field (MRF). This paper addresses semantic segmentation by incorporating high-order relations and mixture of label contexts into MRF. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms. It has several appealing properties. First, different from the recent works that required many iterations of MF during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing models as its special cases. Furthermore, pairwise terms in DPN provide a unified framework to encode rich contextual information in high-dimensional data, such as images and videos. Third, DPN makes MF easier to be parallelized and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated on standard semantic image/video segmentation benchmarks, where a single DPN model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012, Cityscapes dataset and CamVid dataset.

show abstract

Section: Modeling Smoothness Termsmentioning

confidence: 99%

Section: Modeling Smoothness Termsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Markov Random Field for Semantic Segmentation

Liu

Luo

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

143

107

View full text Add to dashboard Cite

show abstract

“…3D convolution: Volumetric (i.e., spatially 3D) convolution has been successfully used in video analysis ( [18]). VoxNet [19] and 3D ShapeNet [20] are two pioneer works in applying 3D convolution on voxelized 3D shapes.…”

Section: Related Workmentioning

confidence: 99%

3D Geometry-Aware Semantic Labeling of Outdoor Street Scenes

Zhong

Dai

2018

2018 24th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

This paper is concerned with the problem of how to better exploit 3D geometric information for dense semantic image labeling. Existing methods often treat the available 3D geometry information (e.g., 3D depth-map) simply as an additional image channel besides the R-G-B color channels, and apply the same technique for RGB image labeling. In this paper, we demonstrate that directly performing 3D convolution in the framework of a residual connected 3D voxel top-down modulation network can lead to superior results. Specifically, we propose a 3D semantic labeling method to label outdoor street scenes whenever a dense depth map is available. Experiments on the "Synthia" and "Cityscape" datasets show our method outperforms the state-ofthe-art methods, suggesting such a simple 3D representation is effective in incorporating 3D geometric information.

show abstract

“…Another group of methods pioneered by [38] predict segmentations from the raw pixels. Methods were introduced to improve the spatial coherence of the semantic segmentation using conditional random fields (CRF) [33,57,9]. Co-segmentation: Co-segmentation was first introduced by [49] for simultaneous binary segmentation of object parts in an image pair.…”

Section: Semantic Segmentationmentioning

confidence: 99%

Semantically Coherent Co-Segmentation and Reconstruction of Dynamic Scenes

Mustafa

Hilton

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

In this paper we propose a framework for spatially and temporally coherent semantic co-segmentation and reconstruction of complex dynamic scenes from multiple static or moving cameras. Semantic co-segmentation exploits the coherence in semantic class labels both spatially, between views at a single time instant, and temporally, between widely spaced time instants of dynamic objects with similar shape and appearance. We demonstrate that semantic coherence results in improved segmentation and reconstruction for complex scenes. A joint formulation is proposed for semantically coherent object-based co-segmentation and reconstruction of scenes by enforcing consistent semantic labelling between views and over time. Semantic tracklets are introduced to enforce temporal coherence in semantic labelling and reconstruction between widely spaced instances of dynamic objects. Tracklets of dynamic objects enable unsupervised learning of appearance and shape priors that are exploited in joint segmentation and reconstruction. Evaluation on challenging indoor and outdoor sequences with hand-held moving cameras shows improved accuracy in segmentation, temporally coherent semantic labelling and 3D reconstruction of dynamic scenes.

show abstract

Feature Space Optimization for Semantic Video Segmentation

Cited by 169 publications

References 22 publications

Deep Learning Markov Random Field for Semantic Segmentation

Deep Learning Markov Random Field for Semantic Segmentation

3D Geometry-Aware Semantic Labeling of Outdoor Street Scenes

Semantically Coherent Co-Segmentation and Reconstruction of Dynamic Scenes

Contact Info

Product

Resources

About