Multimodal CNN Pedestrian Classification: A Study on Combining LIDAR and Camera Data

Melotti, Gledson; Premebida, Cristiano; Gonçalves, Nuno; Nunes, Urbano; Faria, Diego R.

doi:10.1109/itsc.2018.8569666

Cited by 40 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…91,88 Bayesian networks have also been applied to problems in artificial perception 13 and pedestrian classification. 92 For inference, uncertainty about parameters is expressed as a probability distribution over all parameters (parameter set) using Bayes rule (equation 26) PðqjX ½1; X ½2; :::X ½mÞ ¼ PðX ½1; X ½2; :::X ½mjqÞPðqÞ PðX ½1; X ½2; :::X ½mÞ ð26Þ X[i]s are the data samples that are assumed to be independent and identically distributed, and q denotes the parameters of the event of interest. The left hand side of equation (26) defines the posterior distribution.…”

Section: Bayesian Networkmentioning

confidence: 99%

Section: Unsupervised Learningmentioning

confidence: 99%

“…Bayesian networks have also been applied to sensor fusion and place classification in robotics. 94,92…”

Section: Unsupervised Learningmentioning

confidence: 99%

See 2 more Smart Citations

A review of algorithms and techniques for image-based recognition and inference in mobile robotic systems

Tawiah

2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Autonomous vehicles include driverless, self-driving and robotic cars, and other platforms capable of sensing and interacting with its environment and navigating without human help. On the other hand, semiautonomous vehicles achieve partial realization of autonomy with human intervention, for example, in driver-assisted vehicles. Autonomous vehicles first interact with their surrounding using mounted sensors. Typically, visual sensors are used to acquire images, and computer vision techniques, signal processing, machine learning, and other techniques are applied to acquire, process, and extract information. The control subsystem interprets sensory information to identify appropriate navigation path to its destination and action plan to carry out tasks. Feedbacks are also elicited from the environment to improve upon its behavior. To increase sensing accuracy, autonomous vehicles are equipped with many sensors [light detection and ranging (LiDARs), infrared, sonar, inertial measurement units, etc.], as well as communication subsystem. Autonomous vehicles face several challenges such as unknown environments, blind spots (unseen views), non-line-of-sight scenarios, poor performance of sensors due to weather conditions, sensor errors, false alarms, limited energy, limited computational resources, algorithmic complexity, human–machine communications, size, and weight constraints. To tackle these problems, several algorithmic approaches have been implemented covering design of sensors, processing, control, and navigation. The review seeks to provide up-to-date information on the requirements, algorithms, and main challenges in the use of machine vision–based techniques for navigation and control in autonomous vehicles. An application using land-based vehicle as an Internet of Thing-enabled platform for pedestrian detection and tracking is also presented.

show abstract

Section: Bayesian Networkmentioning

confidence: 99%

Section: Unsupervised Learningmentioning

confidence: 99%

See 1 more Smart Citation

A review of algorithms and techniques for image-based recognition and inference in mobile robotic systems

Tawiah

2020

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…Data from a LIDAR enter into the CNN classifier in the form of high-resolution distance/ depth (DM) and reflectance maps (RMs). Distance and intensity (reflectance) raw data from the LIDAR are transformed to high-resolution (dense) maps as described in [9,10].…”

Section: Use Case In Pedestrian Classificationmentioning

confidence: 99%

“…Among several convolutional neural networks, we opted to use AlexNet CNN architecture with batch normalization in the first two layers and the last layer, the softmax activation function with two classes and dropout of 50%. The network was trained from scratch for the pedestrian and nonpedestrian classes [10]. Through the bounding boxes provided by the KITTI dataset, we cropped the objects contained in the depth and reflectance maps images.…”

Section: Use Case In Pedestrian Classificationmentioning

confidence: 99%

Multimodal Bayesian Network for Artificial Perception

Faria

Premebida²,

Manso

et al. 2019

Bayesian Networks - Advances and Novel Applications

Self Cite

View full text Add to dashboard Cite

In order to make machines perceive their external environment coherently, multiple sources of sensory information derived from several different modalities can be used (e.g. cameras, LIDAR, stereo, RGB-D, and radars). All these different sources of information can be efficiently merged to form a robust perception of the environment. Some of the mechanisms that underlie this merging of the sensor information are highlighted in this chapter, showing that depending on the type of information, different combination and integration strategies can be used and that prior knowledge are often required for interpreting the sensory signals efficiently. The notion that perception involves Bayesian inference is an increasingly popular position taken by a considerable number of researchers. Bayesian models have provided insights into many perceptual phenomena, showing that they are a valid approach to deal with real-world uncertainties and for robust classification, including classification in time-dependent problems. This chapter addresses the use of Bayesian networks applied to sensory perception in the following areas: mobile robotics, autonomous driving systems, advanced driver assistance systems, sensor fusion for object detection, and EEG-based mental states classification.

show abstract