The Best of Both Worlds: Combining CNNs and Geometric Constraints for Hierarchical Motion Segmentation

Bideau, Pia; RoyChowdhury, Aruni; Menon, Rakesh R.; Learned-Miller, Erik

doi:10.1109/cvpr.2018.00060

Cited by 43 publications

(53 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, the cat in the middle row is over-segmented by [20] and under-segmented by [5], but well-segmented by our approach. In the final row, both [20] and [5] exhibit segmentation and tracking errors; the region corresponding to the man's foot (colored yellow for Keuper et al and red for Bideau et al) are mistakenly tracked into a background region thus segmenting part of the background as a moving object. Meanwhile, our object-based tracker fully segments the person and the tennis racket with high precision.…”

Section: Comparison To Prior Workmentioning

confidence: 94%

“…In addition to improving segmentation boundaries, our approach effectively removes spurious segmentations of background regions and object parts (Figure 7). Qualitative results: We qualitatively compare our approach with Keuper et al [20] and Bideau et al [5] in Figure 7 2 In the top row of Figure 7, [20] oversegments the dog into multiple parts, and [5] merges the dog with the background, whereas our approach fully segments the dog. 2 [5] only segments objects while they move.…”

Section: Comparison To Prior Workmentioning

confidence: 97%

“…By contrast, our approach directly outputs segmentations at each frame, which we link together with an efficient tracker. Very recently, Bideau et al [5] proposed to combine a heuristicbased motion segmentation method [26,4] with a CNN trained for semantic segmentation for the task of moving object segmentation. Their method, however, does not handle discontinuous motion.…”

Section: Related Workmentioning

confidence: 99%

“…On this proposed metric, we first note that, as ex- Figure 7: Qualitative results comparing our approach to two stateof-the-art methods. Prior work frequently exhibits over-or undersegmentation, such as the cat (middle row, [20]) and the dog (top row, [5]), respectively. Our method fuses motion and appearance information to segment the full extent of moving objects.…”

Section: Comparison To Prior Workmentioning

confidence: 99%

See 3 more Smart Citations

Towards Segmenting Anything That Moves

Dave

Tokmakov

Ramanan

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Detecting and segmenting individual objects, regardless of their category, is crucial for many applications such as action detection or robotic interaction. While this problem has been well-studied under the classic formulation of spatio-temporal grouping, state-of-the-art approaches do not make use of learning-based methods. To bridge this gap, we propose a simple learning-based approach for spatiotemporal grouping. Our approach leverages motion cues from optical flow as a bottom-up signal for separating objects from each other. Motion cues are then combined with appearance cues that provide a generic objectness prior for capturing the full extent of objects. We show that our approach outperforms all prior work on the benchmark FBMS dataset. One potential worry with learning-based methods is that they might overfit to the particular type of objects that they have been trained on. To address this concern, we propose two new benchmarks for generic, moving object detection, and show that our model matches topdown methods on common categories, while significantly out-performing both top-down and bottom-up methods on never-before-seen categories.

show abstract

Section: Comparison To Prior Workmentioning

confidence: 94%

Section: Comparison To Prior Workmentioning

confidence: 97%

Section: Related Workmentioning

confidence: 99%

Section: Comparison To Prior Workmentioning

confidence: 99%

See 2 more Smart Citations

Towards Segmenting Anything That Moves

Dave

Tokmakov

Ramanan

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…In addition constraints on the occlusion regions [27] and discontinuities [14] have been used. Recently, machine learning techniques have been used for motion segmentation [13], [5]. As discussed next, the well-known SfM learner acquires both, the depth map and the rigid camera motion, and thus the flow due to rigid motion is fully constrained.…”

Section: B Independent Motion Detectionmentioning

confidence: 99%

EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras

Mitrokhin

Fermüller

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We present the first event-based learning approach for motion segmentation in indoor scenes and the first eventbased dataset -EV-IMO -which includes accurate pixelwise motion masks, egomotion and ground truth depth. Our approach is based on an efficient implementation of the SfM learning pipeline using a low parameter neural network architecture on event data. In addition to camera egomotion and a dense depth map, the network estimates independently moving object segmentation at the pixel-level and computes per-object 3D translational velocities of moving objects. We also train a shallow network with just 40k parameters, which is able to compute depth and egomotion.Our EV-IMO dataset features 32 minutes of indoor recording with up to 3 fast moving objects in the camera field of view. The objects and the camera are tracked using a VICON R motion capture system. By 3D scanning the room and the objects, ground truth of the depth map and pixel-wise object masks are obtained. We then train and evaluate our learning pipeline on EV-IMO and demonstrate that it is well suited for scene constrained robotics applications. SUPPLEMENTARY MATERIALThe supplementary video, code, trained models and appendix will be made available at

show abstract