A Joint 3D-2D Based Method for Free Space Detection on Roads

Abstract

In this paper, we address the problem of road segmentation and free space detection in the context of autonomous driving. Traditional methods either use 3-dimensional (3D) cues such as point clouds obtained from LIDAR, RADAR or stereo cameras or 2-dimensional (2D) cues such as lane markings, road boundaries and object detection. Typical 3D point clouds do not have enough resolution to detect fine differences in heights such as between road and pavement. Image based 2D cues fail when encountering uneven road textures such as due to shadows, potholes, lane markings or road restoration. We propose a novel free road space detection technique combining both 2D and 3D cues. In particular, we use CNN based road segmentation from 2D images and plane/box fitting on sparse depth data obtained from SLAM as priors to formulate an energy minimization using conditional random field (CRF), for road pixels classification. While the CNN learns the road texture and is unaffected by depth boundaries, the 3D information helps in overcoming texture based classification failures. Finally, we use the obtained road segmentation with the 3D depth data from monocular SLAM to detect the free space for the navigation purposes. Our experiments on KITTI odometry dataset [12], Camvid dataset [7] as well as videos captured by us validate the superiority of the proposed approach over the state of the art.

A Multimodal Perception-Driven Self Evolving Autonomous Ground Vehicle

“…Unlike range data, image data is rich in context, providing a large amount of information over a broad area. Most contemporary FSD techniques focus on CNN and large quantities of data [18], [44]. When annotated data is lacking, a CNN will not generalize adequately, overfit the model, and misclassifying traversable space.…”

Free Space Detection Algorithm Using Object Tracking for Autonomous Vehicles

“…Recently, various studies on autonomous vehicles have been conducted in various fields. There are papers on various signal processing methods of LiDAR sensors mainly used in autonomous vehicles [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 ]. Among them, there are many papers that presented research on free space sensing.…”

“…Free space detection is being studied not only for the study of autonomous vehicle driving, but also in various fields such as the recognition of a parking space or the driving of an aircraft [ 25 , 26 , 27 ]. Among them, the research trends in the detection of free space in autonomous vehicles are as follows [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. Often, a camera or LiDAR sensor is used, or both sensors are used, to recognize the space and obstacles around the vehicle.…”

“…J. Tao [ 3 ] used LiDAR sensor data to detect free space, in which free space was also detected without information about the speed of the obstacle. Many previous studies used static data without considering the speed of obstacles [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. Free space detection for autonomous driving must also be considered in dynamic environments.…”

“…Table 1 summarizes existing papers [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. In previous papers, sensors were used in the FSD process, and the speed information of obstacles was summarized.…”