Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues

Gupta, Abhishek; Anpalagan, Alagan; Guan, Ling; Khwaja, Ahmed Shaharyar

doi:10.1016/j.array.2021.100057

Cited by 316 publications

(112 citation statements)

References 171 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A comprehensive survey of DL applications for object detection and scene perception in autonomous vehicles applicable to the perception domain is provided by Gupta et al (2021). The paper examines the theory underlying self-driving vehicles from the DL perspective and current implementations, followed by critical evaluations.…”

Section: Ai Based Techniquesmentioning

confidence: 99%

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

Vermesan

John

Pype

et al. 2021

Front. Future Transp.

View full text Add to dashboard Cite

The automotive sector digitalization accelerates the technology convergence of perception, computing processing, connectivity, propulsion, and data fusion for electric connected autonomous and shared (ECAS) vehicles. This brings cutting-edge computing paradigms with embedded cognitive capabilities into vehicle domains and data infrastructure to provide holistic intrinsic and extrinsic intelligence for new mobility applications. Digital technologies are a significant enabler in achieving the sustainability goals of the green transformation of the mobility and transportation sectors. Innovation occurs predominantly in ECAS vehicles’ architecture, operations, intelligent functions, and automotive digital infrastructure. The traditional ownership model is moving toward multimodal and shared mobility services. The ECAS vehicle’s technology allows for the development of virtual automotive functions that run on shared hardware platforms with data unlocking value, and for introducing new, shared computing-based automotive features. Facilitating vehicle automation, vehicle electrification, vehicle-to-everything (V2X) communication is accomplished by the convergence of artificial intelligence (AI), cellular/wireless connectivity, edge computing, the Internet of things (IoT), the Internet of intelligent things (IoIT), digital twins (DTs), virtual/augmented reality (VR/AR) and distributed ledger technologies (DLTs). Vehicles become more intelligent, connected, functioning as edge micro servers on wheels, powered by sensors/actuators, hardware (HW), software (SW) and smart virtual functions that are integrated into the digital infrastructure. Electrification, automation, connectivity, digitalization, decarbonization, decentralization, and standardization are the main drivers that unlock intelligent vehicles' potential for sustainable green mobility applications. ECAS vehicles act as autonomous agents using swarm intelligence to communicate and exchange information, either directly or indirectly, with each other and the infrastructure, accessing independent services such as energy, high-definition maps, routes, infrastructure information, traffic lights, tolls, parking (micropayments), and finding emergent/intelligent solutions. The article gives an overview of the advances in AI technologies and applications to realize intelligent functions and optimize vehicle performance, control, and decision-making for future ECAS vehicles to support the acceleration of deployment in various mobility scenarios. ECAS vehicles, systems, sub-systems, and components are subjected to stringent regulatory frameworks, which set rigorous requirements for autonomous vehicles. An in-depth assessment of existing standards, regulations, and laws, including a thorough gap analysis, is required. Global guidelines must be provided on how to fulfill the requirements. ECAS vehicle technology trustworthiness, including AI-based HW/SW and algorithms, is necessary for developing ECAS systems across the entire automotive ecosystem. The safety and transparency of AI-based technology and the explainability of the purpose, use, benefits, and limitations of AI systems are critical for fulfilling trustworthiness requirements. The article presents ECAS vehicles’ evolution toward domain controller, zonal vehicle, and federated vehicle/edge/cloud-centric based on distributed intelligence in the vehicle and infrastructure level architectures and the role of AI techniques and methods to implement the different autonomous driving and optimization functions for sustainable green mobility.

show abstract

Section: Ai Based Techniquesmentioning

confidence: 99%

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

Vermesan

John

Pype

et al. 2021

Front. Future Transp.

View full text Add to dashboard Cite

show abstract

“…The late fusion or decision fusion combines the unimodel decisions to conclude the final decision [20]. The late-fusion models are more accessible to simulate than early fusion models, especially when the modalities have inconstant sampling rate and data dimensionality [21,22]. In [23], a feature transformation network has been designed to learn the RGB and depth features.…”

Section: Data Fusion Importancementioning

confidence: 99%

Deep Learning Inspired Object Consolidation Approaches Using LiDAR Data for Autonomous Driving: A Review

Mekala

Park

Dhiman³

et al. 2021

Arch Computat Methods Eng

View full text Add to dashboard Cite

Autonomous Driving Vehicle (ADV) services have become a prominent motif in intelligent vehicle technology by adapting deep learning features. Automated driverless services are a hercules task due to the dynamic driving environment and the performance is deliberately reliant on the quality of data fusion from sensors. Therefore, considering advanced 3D LiDAR sensors is essential to measure the surrounding with 360 • coverage. However, accomplishing maximum autonomy is the main challenge because of debilitated and complex driving environmenst. Deep Learning (DL) based models potentially impact surrounding measures for object detection, classification, and tracking. This study describes the importance of DL-LiDAR strategies to formulate ADV research challenges followed by a comprehensive analysis of Semantic Segmentation, Data Fusion, Data Representation, Feature Extraction, Dynamic Object Detection, and Autonomous Driving-Multi-Objective Tracking mechanisms. Unlike existing review papers, we examine various deep learning influenced approaches and describe their precarious measurements. Also, we analyze the impact of systematic methods based on open-source data sets for objectx (x=classification, localization, tracking) consolidation. Most of the summaries confine scalability, and feasibility of DL techniques in ADV to accomplish an intelligent driving system without human interference. Open research problems are discussed in the last section of the article.

show abstract

“…Podsumowując, metody mające na celu usprawnienie jakości działania kamery w trudnych warunkach atmosferycznych, można podzielić na: próby skonstruowania specjalnego, dedykowanego tym zadaniom sprzętu i metody oparte o poprawę jakości analizy obrazu. W przy przypadku tych drugich metod, najlepiej rokujące są sztuczne sieci neuronowe [20,32]. Sieci te wykorzystuje się także do łączenia informacji płynących z różnych sensorów aby w ten sposób zwiększyć jakość detekcji i jej dokładność [7].Warto zauważyć, że wpływ warunków atmosferycznych na pracę kamery jest na tyle istotny, że podejmuje się wiele prób tworzenia baz danych i wyszukanych metod ewaluacji, tak aby zapewnić możliwie rzeczywiste warunki do testowania nowych rozwiązań nie wymagających każdorazowego budowania od podstaw skomplikowanych stanowisk badawczych [9,22].…”

Section: Kameraunclassified

“…Innym sposobem jest wykorzystanie algorytmów pozwalających na lepszą interpretację uzyskanych danych. Rozwój sieci neuronowych sprzyja tworzeniu systemów wykorzystujących łączenie różnego rodzaju czujników, ponieważ to właśnie na nich najczęściej opiera się ich architektura [20,39]. Potwierdzają to wyniki badań eksperymentalnych przeprowadzonych w Seulu, polegających na wykorzystaniu kamery i LiDARu jako rozwiązania problemu poruszania się samochodu autonomicznego w deszczu.…”

Section: Podsumowanieunclassified

Wpływ Niekorzystnych Warunków Atmosferycznych Na Działanie Sensorów Wykorzystywanych W Samochodach Autonomicznych

BRZOZOWSKI¹,

Parczewski²

2021

Projektowanie, Badania I Eksploatacja

View full text Add to dashboard Cite

Streszczenie: Celem artykułu jest objaśnienie wpływu niekorzystnych warunków pogodowych na najbardziej popularne sensory wykorzystywane w samochodach autonomicznych. W pracy przedstawiono typowe sensory takie jak: kamera, LiDAR oraz RADAR, a następnie wskazano zakłócenia jakie w ich pracy wywołują niekorzystne warunki pogodowe. Przedstawiono wpływ: deszczu, śniegu oraz mgły. Ponadto omówiono próby zmierzające do ograniczenia tego wpływu. Proponowane działania dotyczą zarówno usprawnień samego sprzętu jak i rozwijania metod analizy obrazu. Podsumowaniem opracowania jest konkluzja, że najlepsze efekty otrzymuje się podczas wykorzystywania różnych sensorów, które mogą się wzajemnie uzupełniać.

show abstract

Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues

Cited by 316 publications

References 171 publications

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

Deep Learning Inspired Object Consolidation Approaches Using LiDAR Data for Autonomous Driving: A Review

Wpływ Niekorzystnych Warunków Atmosferycznych Na Działanie Sensorów Wykorzystywanych W Samochodach Autonomicznych

Contact Info

Product

Resources

About