Computer Vision System with 2D and 3D Data Fusion for Detection of Possible Auxiliaries Routes in Stretches of Interdicted Roads

Abstract

In this paper we present an intelligent system to help autonomous vehicles in real cities and with local traffic rules. A 2D and 3D visual attention system is proposed, capable of detecting the use of signs and aids in cases of major roadblock (road under work, with a traffic accident, etc.). For this to be possible, we analyze the cones and traffic signs that usually alert a driver about this type of problem. The main objective is to provide support for autonomous vehicles to be able to find an auxiliary route that is not previously mapped. For this we use a Grid Point Cloud Map. Using the ORB-SLAM visual odometry system we can correctly fit each stereo frame point cloud in the pose where the images were collected. With the concatenation of point clouds generated by the stereo camera, every grid block can draw the main characteristics of its region and an auxiliary route can be mapped. In this type of situation the vision system must work in real time. The results are promising and very satisfactory, we obtained an accuracy of 98.4% in the 2D classification task and 83% accuracy in the single frame 3D detection task.

Research Scenarios of Autonomous Vehicles, the Sensors and Measurement Systems Used in Experiments

“…These scenarios are focused on the cognitive aspect of experimental research. Such scenarios are also needed during component/system research, e.g., object detection by the perception system [67][68][69], path planning systems [19,33], and road line/edge detection [26,27,70]. Nonnormative AV test scenarios were analyzed.…”

“…− 3D object detection on the road [72,76] using lidar and sensor fusion (e.g., lidar and camera), where the aim is to obtain information about the obstacle in the spatial coordinate system: car detection [19,75,77]; pedestrian detection [23,24,78]; bicyclist/motorcyclist detection [23,25,73]; cone detection [67]; detection of walls, trees, bushes, and poles [78]; detection and tracking of unknown objects [79].…”

“…− buses [79]; − trucks [67]; − microcars (single person) [34]; − go-karts [32]; − research platforms (autonomous systems carriers) or physical models of the car at a reduced scale [80,81].…”

“…If the course of the research is not related to the realization of critical maneuvers, the scenarios depend on the state of road traffic. Research results according to such scenarios are available in [25,67,70].…”

“…Most solutions based on vision sensors use single-lens cameras that provide a 2D image for the road situation analysis system [24,32,72]. There are also vehicles and systems that use stereo cameras [31,67,74]. These sensors provide an image from two lenses, and this creates an image with depth.…”