CNN‐Transformer for visual‐tactile fusion applied in road recognition of autonomous vehicles

Shi, Runwu; Yang, Shichun; Chen, Yuyi; Wang, Rui; Zhang, Mengyue; Lu, Jiayi; Cao, Yaoguang

doi:10.1016/j.patrec.2022.11.023

Cited by 22 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even in areas with robust infrastructure, there remain data and knowledge gaps, primarily due to the unpredictable nature of agricultural environments [146]. Addressing the issue of missing data modalities, especially in the visual and tactile domains, remains a significant challenge [144].…”

Section: Discrepancies In Data Characteristics and Modalitiesmentioning

confidence: 99%

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Fahmy,

Hassan,

Hussain

et al. 2024

Preprint

View full text Add to dashboard Cite

The integration of artificial intelligence with sensor technologies has revolutionized precision agriculture, offering unprecedented opportunities for enhancing crop management and productivity. This review focuses on the latest advancements in vision-based tactile sensors, a technology at the forefront of this transformation. By combining tactile data with vision-based techniques, these sensors provide a more comprehensive understanding of the agricultural environment. We investigate thoroughly the role of deep learning approaches in refining the functionality of these sensors, highlighting their potential to significantly improve the accuracy and efficiency of agricultural operations. The paper also explores the importance of specialized datasets in training deep neural networks for vision-based tactile applications, assessing the current landscape and identifying gaps in the available data. Through a thorough examination of the current state of the art, this review paper aims to shed light on the potential of AI-driven tactile sensing in precision agriculture and outline future research directions to further advance this field.

show abstract

Section: Discrepancies In Data Characteristics and Modalitiesmentioning

confidence: 99%

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Fahmy,

Hassan,

Hussain

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…2. 1 Generally, neural network score values are obtained using a prediction function that normalizes logit values between zero and one, such as the softmax prediction function. The curves on the left hand-side of Fig.…”

Section: Kernel Density Estimation and Normalized Histogrammentioning

confidence: 99%

“…Research on sensory perception has achieved very satisfactory results in terms of object recognition, contributing significantly to the progress of autonomous and intelligent vehicles (AV/IV) and robotics, due to technological advances such as hardware, sensors and statistical learning techniques [1]- [3]. Perception systems for AV/IV can be understood as a process that interprets the data provided by the sensors in order to understand the surrounding environment, thus contributing to safer decision-making.…”

Section: Introductionmentioning

confidence: 99%

Reducing Overconfidence Predictions in Autonomous Driving Perception

Melotti,

Premebida,

Bird

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Object recognition is a crucial step in perception systems for autonomous and intelligent vehicles, as evidenced by the numerous research works in the topic. In this paper, object recognition is explored by using multisensory and multimodality approaches, with the intention of reducing the false positive rate (FPR). The reduction of the FPR becomes increasingly important in perception systems since the misclassification of an object can, depending on the circumstances, potentially cause accidents. In particular, this work presents a strategy through Bayesian inference to reduce the FPR considering the likelihood function as a cumulative distribution function from Gaussian kernel density estimations, and the prior probabilities as cumulative functions of normalized histograms. The validation of the proposed methodology is performed on the KITTI dataset using deep networks (DenseNet, NasNet, and EfficientNet), and recent 3D point cloud networks (PointNet, and PointNet++), by considering three object-categories (cars, cyclists, pedestrians) and the RGB and LiDAR sensor modalities.

show abstract

“…Sensors of intelligent driving vehicles include cameras, lidar, millimeter-wave radar, ultrasonic radar, etc. [6] . Among them, cameras become the most commonly used sensors in intelligent driving vehicles due to their high information content, low cost and operation power [7] .…”

Section: Introductionmentioning

confidence: 99%

Research on sensor dirt recognition of autonomous vehicle camera based on deep learning

Wu,

Wang,

Ran

et al. 2024

Second International Conference on Informatics, Networking, and Computing (ICINC 2023)

View full text Add to dashboard Cite

Intelligent driving technology is an important direction of the current development of automobile technology. The realization of intelligent technology is based on the perception ability of car sensors such as cameras and lidar for the driving environment detection. In the actual driving, the surface pollution of sensors will reduce the safety of intelligent driving vehicles. Therefore, it is very important to realize the automatic recognition of ultra-near-field dirt on the optical surface of car sensors. At present, the research on camera sensor fouling recognition is very scarce, and the relevant public data set for recognition research is also lacking. This paper conducted an in-depth study on the contamination problem of the camera sensor of the autonomous vehicle, independently created a dataset of dirt on the camera sensor, and trained the dataset based on the YOLO-V5 model using this dataset. The results show that the dataset we constructed is quite reasonable, with obvious features and excellent training results, with mAP(0.5) reaching 0.987; The results of the detection experiment prove that the trained dirt recognition model has certain universality, and has relatively excellent detection effect on different datasets, and the accuracy rate and recall rate can reach more than 0.8. This work could provide a certain reference for the cleaning research of automatic driving camera sensor.

show abstract

CNN‐Transformer for visual‐tactile fusion applied in road recognition of autonomous vehicles

Cited by 22 publications

References 25 publications

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Vision-Based Tactile Sensors in Precision Agriculture: Deep Learning Approaches, Applications, and Limitations

Reducing Overconfidence Predictions in Autonomous Driving Perception

Research on sensor dirt recognition of autonomous vehicle camera based on deep learning

Contact Info

Product

Resources

About