End-to-End Deep Learning Applied in Autonomous Navigation using Multi-Cameras System with RGB and Depth Images

Díaz-Amado, José; Gomes, Iago Pachêco; Amaro, Jean; Wolf, Denis F.; Osório, Fernando Santos

doi:10.1109/ivs.2019.8813897

Cited by 6 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This allows the model to anticipate changes in the scene and adjust the steering angle, accordingly, enhancing the vehicle's performance and safety. Further, A system for autonomous navigation employing DL and a multi-camera configuration with RGB and depth pictures was proposed by José A. Diaz Amado et al [61]. The CNN employed in the neural network produces the vehicle's steering and throttle commands by using RGB and depth pictures as inputs.…”

Section: Detailed Survey On End-to-end Learning For Sdvsmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

Ganesan,

Kandhasamy,

Chokkalingam

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Self-Driving Vehicles (SDVs) are increasingly popular, with companies like Google, Uber, and Tesla investing significantly in self-driving technology. These vehicles could transform commuting, offering safer, and efficient transport. A key SDV aspect is motion planning, generating secure, and efficient routes. This ensures safe navigation and prevents collisions with obstacles, pedestrians, and other vehicles. Deep Learning (DL) could aid SDV motion planning. AI tools and algorithms, like Artificial Neural Networks (ANNs), Machine Learning (ML) and DL can learn from data to create effective driving strategies, enhancing SDV adaptability to changing conditions for improved safety and efficiency. This survey gives a DL-based motion planning overview for SDVs, covering behaviour planning, trajectory planning, and End to End Learning (E2EL). It assesses various DL-based behaviour and trajectory planning methods, comparing and summarizing them. It also reviews diverse E2EL techniques including Imitation Learning (IL) and Reinforcement Learning (RL) gaining traction lately. Additionally, this review emphasizes the significance of two crucial enablers: datasets and simulation deployment frameworks for SDVs. The survey compares strategies using multiple metrics and highlights DL-based SDV implementation challenges, including simulation and real-world use cases. This article also suggests future research directions to address E2EL and DL-based motion planning limitations. The presented article is an excellent reference for scholars, engineers, and decision-makers who have an interest in DL-based SDV motion planning.

show abstract

Section: Detailed Survey On End-to-end Learning For Sdvsmentioning

confidence: 99%

“…Tinghan Wang [70], Satya R. Jaladi [73] Zhengyuan Yang [57], Tanmay Vilas Samak [64] ,Jie Hu [71], Lei Han [72], Nguyen Thi Hoai Thu [75], Oskar Natan [76] Junekyo Jhung [58], Tianhao Wu [60], José A. Diaz Amado [61],…”

Section: Simulation With Real Worldmentioning

confidence: 99%

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

Ganesan,

Kandhasamy,

Chokkalingam

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Weight of Rovers (kg) 5: Highways and traffic road [44] Racing track on traffic road 210 1231 [40] Highways (sunlight facing the camera, high contrast sunlight, shadows, covered in snow) 14,000 1579 [37] Traffic road 380 - [38] Traffic road and walkways in parks 8400 -4: Off-road on cemented paths, short grass, pebbles, dirt and dry leaves [41] Off-road racing track 300 22 [42] Mowed and short grass off-trail 380 35 [43] Cemented and off-road trails with pebbles, dirt, sand, grass and fallen leaves, with few obstacles 5500 35 3: Sidewalks and walkways in urban environment [47] Static environments: walkways in office areas, laboratory space and corridors; dynamic environments: sidewalks among crowds. 355 12 [45] Paved road cemented on grass 800 50 [46] Mowed lawn, short grass, and trees in urban environment 1000 17 [53] Sidewalks outside malls and office buildings 5500 - [48] Walkways in neighborhoods and parks 60 5 [39] Parking lots, city roads and sidewalks 60 62 [25] Corridor indoors and stone trail outdoors 400 30…”

Section: Approximate Sensors Cost (Gbp)mentioning

confidence: 99%

End-to-End Learning for Visual Navigation of Forest Environments

Niu

Zauner

Tarapore

2023

Forests

View full text Add to dashboard Cite

Off-road navigation in forest environments is a challenging problem in field robotics. Rovers are required to infer their traversability over a priori unknown and dynamically changing forest terrain using noisy onboard navigation sensors. The problem is compounded for small-sized rovers, such as that of a swarm. Their size-proportional low-viewpoint affords them a restricted view for navigation, which may be partially occluded by forest vegetation. Hand-crafted features, typically employed for terrain traversability analysis, are often brittle and may fail to discriminate obstacles in varying lighting and weather conditions. We design a low-cost navigation system tailored for small-sized forest rovers using self-learned features. The MobileNet-V1 and MobileNet-V2 models, trained following an end-to-end learning approach, are deployed to steer a mobile platform, with a human-in-the-loop, towards traversable paths while avoiding obstacles. Receiving a 128 × 96 pixel RGB image from a monocular camera as input, the algorithm running on a Raspberry Pi 4, exhibited robustness to motion blur, low lighting, shadows and high-contrast lighting conditions. It was able to successfully navigate a total of over 3 km of real-world forest terrain comprising shrubs, dense bushes, tall grass, fallen branches, fallen tree trunks, and standing trees, in over five different weather conditions and four different times of day.

show abstract

“…This type of architecture uses a single data source for the input layer to generate the setpoints directly for the control elements of the vehicle. The SiD architectures use the visual information provided by one or more cameras located on the front and periphery of the vehicle to compose a single image of the vehicles field of view of the vehicle as a visual input to the network [15,29,30]. Before being processed by the DNN, the images are reduced in size and normalized.…”

Section: Sid-e2e Architecturementioning

confidence: 99%

“…Mixed data architectures allow different data sources from the vehicle, such as RADAR, longitudinal and lateral accelerations, angular velocities, maps or GPS to be merged together with the visual information from the vehicle's cameras. The inclusion of more information sources in the DNN aims to: (1) improve the performance of the model, (2) improve the prediction of specific cases or abnormal driving; and (3) increase the tolerance to failures produced by the data sources [21,29,33]. As shown in Figure 5, this type of architecture combines the results of the SiD-e2e, such as those shown in the previous Section 2.3.1, with a set of FC layers which allows the mapping of the characteristics from other vehicle data sources on a layer that concatenates all the information.…”

Section: Mid-e2e Architecturementioning

confidence: 99%

End-to-End Deep Neural Network Architectures for Speed and Steering Wheel Angle Prediction in Autonomous Driving

et al. 2021

View full text Add to dashboard Cite

The complex decision-making systems used for autonomous vehicles or advanced driver-assistance systems (ADAS) are being replaced by end-to-end (e2e) architectures based on deep-neural-networks (DNN). DNNs can learn complex driving actions from datasets containing thousands of images and data obtained from the vehicle perception system. This work presents the classification, design and implementation of six e2e architectures capable of generating the driving actions of speed and steering wheel angle directly on the vehicle control elements. The work details the design stages and optimization process of the convolutional networks to develop six e2e architectures. In the metric analysis the architectures have been tested with different data sources from the vehicle, such as images, XYZ accelerations and XYZ angular speeds. The best results were obtained with a mixed data e2e architecture that used front images from the vehicle and angular speeds to predict the speed and steering wheel angle with a mean error of 1.06%. An exhaustive optimization process of the convolutional blocks has demonstrated that it is possible to design lightweight e2e architectures with high performance more suitable for the final implementation in autonomous driving.

show abstract

End-to-End Deep Learning Applied in Autonomous Navigation using Multi-Cameras System with RGB and Depth Images

Cited by 6 publications

References 11 publications

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

A Comprehensive Review on Deep Learning-Based Motion Planning and End-to-End Learning for Self-Driving Vehicle

End-to-End Learning for Visual Navigation of Forest Environments

End-to-End Deep Neural Network Architectures for Speed and Steering Wheel Angle Prediction in Autonomous Driving

Contact Info

Product

Resources

About