MODNet: Motion and Appearance based Moving Object Detection Network for Autonomous Driving

Abstract

For autonomous driving, moving objects like vehicles and pedestrians are of critical importance as they primarily influence the maneuvering and braking of the car. Typically, they are detected by motion segmentation of dense optical flow augmented by a CNN based object detector for capturing semantics. In this paper, our aim is to jointly model motion and appearance cues in a single convolutional network. We propose a novel two-stream architecture for joint learning of object detection and motion segmentation. We designed three different flavors of our network to establish systematic comparison. It is shown that the joint training of tasks significantly improves accuracy compared to training them independently. Although motion segmentation has relatively fewer data than vehicle detection. The shared fusion encoder benefits from the joint training to learn a generalized representation. We created our own publicly available dataset (KITTI MOD) by extending KITTI object detection to obtain static/moving annotations on the vehicles. We compared against MPNet as a baseline, which is the current state of the art for CNN-based motion detection. It is shown that the proposed two-stream architecture improves the mAP score by 21.5% in KITTI MOD. We also evaluated our algorithm on the non-automotive DAVIS dataset and obtained accuracy close to the state-of-the-art performance. The proposed network runs at 8 fps on a Titan X GPU using a basic VGG16 encoder.

Spherical formulation of moving object geometric constraints for monocular fisheye cameras

“…It's hard to give an objective comparison against state of the art, as we are proposing a method to work on fisheye cameras. No public automotive fisheye dataset exists with appropriate ground truth (although we acknowledge that the fisheye data augmentation on existing large-scale datasets [27] may alleviate the issue), and existing methods [10], [11], [25] are designed to work on standard field of view cameras. However, if we observe the published results of MODNet [11] we can see that it can sometimes suffer from similar false positives as our proposal, for example as shown in Figure 9 1 .…”

“…There has also been very promising work in using Convolutional Neural Networks (CNN) to solve the moving object detection problem (e.g. MODNet [11], MPNet [12]). However, these methods require large annotated datasets to make it scene agnostic and it is difficult to ensure it detects objects purely based on motion cues and not overfit to appearance cues of commonly occurring moving objects like vehicles or pedestrians.…”

Mining Minimal Map-Segments for Visual Place Classifiers

“…For trajectory estimation, the KLT tracker [5] was adopted. KLT is one of most widely used optical flow estimation techniques in robotics [19] and autonomous driving [20]. Formally, N = 1, 500 features at the initial frame are created and tracked over successive frames.…”

Cascaded Cross-Modality Fusion Network for 3D Object Detection

“…Among various tasks of scene understanding, object detection is crucial for autonomous driving [ 1 ], robotics, and augmented reality. Deep learning-based 2D object detection which aims to predict the position and category of targets with given images has made unprecedented achievement in recent years [ 2 ].…”