Latest Advancements in Perception Algorithms for ADAS and AV Systems Using Infrared Images and Deep Learning

Abstract

Perception system plays an important role in Advanced driver assistance systems (ADAS) & Autonomous vehicles (AV) to understand the surrounding environment and further navigation. It is highly challenging to achieve the accurate perception of ego vehicle mimicking human vision. The available ADAS and AV solutions could able to perceive the environment to some extent using multiple sensors like Lidars, Radars and Cameras. National Highway Traffic Safety Administration Crash reports of ADAS and AV systems shows that the complete autonomy is challenging to achieve using the existing sensor suite. Particularly, in extreme weather, low light and night scenarios, there is a need for additional perception sensors. Infrared camera seems to be one of the potential sensors to address such extreme and corner cases. This chapter aimed to discuss the advantage of adding infrared sensors to perceive the environment accurately. The advancements in deep learning approaches further leverages to enhance ADAS features. Also, the limitations of current sensors, the need for infrared sensors and technology, artificial intelligence and current research focus using IR images are discussed in detail. Literature shows that by adding IR sensor to existing sensor suite may lead a way to achieve level 3 and above autonomous driving precisely.