An Extensible Multi-Sensor Fusion Framework for 3D Imaging

Siddiqui, Talha Ahmad; Madhok, Rishi; O’Toole, Matthew

doi:10.1109/cvprw50498.2020.00512

Cited by 8 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Cao et al, 2018) show that the collaborative fusion of the different sources via a CNN and the use of semantic features makes it possible to obtain better image semantic segmentation results. (Siddiqui et al, 2020) use three data sources (lidar, single photon lidar and camera) to improve data semantic segmentation for autonomous cars. use two data sources (lidar and hyperspectral imagery) to better classify the points and pixels in their data.…”

Section: Multi-source Semantic Segmentationmentioning

confidence: 99%

Multi-Source Point Cloud Semantic Segmentation Using Neural Network

Montlahuc

Polette

Tahan

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. The purpose of this study is to enhance point cloud semantic segmentation by using point clouds from multiple distinct technologies on the same capture location and to determine whether employing various technologies throughout the acquisition process yields better performance during classification. The different point clouds were captured in the same geographical location and have previously been aligned and classified by professionals of the field. Three locations have been scanned with airborne lidar, terrestrial lidar and photogrammetry using UAV or helicopter. The use of various sources of capture on the same location opens the door to creating new features, such as the proportion of each source involved in the semantic segmentation of point clouds. This plurality of sources also enables us to spread various features, such as RGB colors, that have been propagated to other sources via the neighborhood. The initial results lean towards capture using different technologies as the overall accuracy increase by two to four points and the mean Matthews correlation coefficient increase by four to seven points. The main drawbacks are the cost of some technologies, as well as the processing time, which is greater than with a single technology.

show abstract

Section: Multi-source Semantic Segmentationmentioning

confidence: 99%

Multi-Source Point Cloud Semantic Segmentation Using Neural Network

Montlahuc

Polette

Tahan

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Interpreting spatio-temporal data visualizations is an example of a larger research problem in vehicle telemetry analytics and the development of multi-sensor autonomous vehicles. Research on data visualization techniques for developing multi-sensor vehicles involves both autonomous vehicle operation systems [6] [7] and human analyst systems [8] [9] where one goal is to minimize the visual interference that occurs when working with real-time multi-sensor data. In the context of human analyst systems there are two forms of visual interference, occlusion [10] [11] and loss of context when zooming in to reveal detail [12].…”

Section: Introductionmentioning

confidence: 99%

A Visual Decision-Support System using Fingerprint Matrices applied to Cyclical Spatio-Temporal Data from Motorsports

Street¹,

Hoeber²

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Visualizing cyclical spatio-temporal data is an important part of understanding how and why objects move in the context of motorsports, which is critical feedback for drivers to improve their performance. Current methods have problems such as occlusion and loss of context which significantly limit our ability to see and understand vehicle data. Here we demonstrate how the fingerprint matrix method (which is normally used in lexical analysis) can be applied in vehicle motion analysis to overcome these two problems. Compared to traditional methods using traction circle scatterplot displays of acceleration force data from a race car, our prototype design allows decision makers to see individual datapoints in a more concise display. We show that informative but previously-hidden anomalies and patterns become more easily recognized in the data. Our design generalizes to other cyclical spatio-temporal visualization problems involving transportation, medicine, and the natural world.

show abstract

“…The challenge of depth recovery is being tackled in many ways with data fusion techniques being used to combine single pixel [19,26,24,11], dual pixel [11] and SPAD arrays [23,15]. However, we are approaching this challenge in a different way with an emphasis on a compact solution with new applications for wearable tech, drones and mobility vehicles.…”

mentioning

confidence: 99%

Efficient Bayesian deep inversion

Higham,

Johnson,

Radwell

et al. 2023

JCD

View full text Add to dashboard Cite

We develop a deep learning method to enhance sensor detection for depth prediction. Our novel system combines sensor hardware and Bayesian inference to solve the underlying inverse problem, recovering depth from measurements. The hardware comprises single sensor non-scanning time-of-flight laser detection with synchronised video to produce a 3D depth map. The Bayesian framework provides depth prediction with uncertainty quantification. A conditional generator-discriminator adversarial network is adapted to learn a compact representation of the scene that recovers 3D depth at 30 Hz using a large training set. We transfer the network to a real hardware system and compare with ground truth depth information. Our novel synthesis of hardware and machine learning technologies addresses the important challenge of providing accurate absolute depth prediction at video rate with efficient and cost-effective non-scanning laser detection technology. This flexible and compact system has many exciting applications for autonomous vehicles, drones and wearable technology.

show abstract

An Extensible Multi-Sensor Fusion Framework for 3D Imaging

Cited by 8 publications

References 32 publications

Multi-Source Point Cloud Semantic Segmentation Using Neural Network

Multi-Source Point Cloud Semantic Segmentation Using Neural Network

A Visual Decision-Support System using Fingerprint Matrices applied to Cyclical Spatio-Temporal Data from Motorsports

Efficient Bayesian deep inversion

Contact Info

Product

Resources

About