Disparity Map Estimation From Cross-Modal Stereo

Rukkanchanunt, Thapanapong; Shibata, Takashi; Tanaka, Masayuki; Okutomi, Masatoshi

doi:10.1109/globalsip.2018.8646523

Cited by 2 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… – 13 , 23 – 29 The main challenge in the early days was how to combine and make use of information from both modalities 30 – 33 Most existing methods assume that visible-thermal image pairs are geometrically aligned. Those methods fuse both modalities’ features directly in corresponding pixel positions, as shown in Fig.…”

Section: Related Workmentioning

confidence: 99%

Multi-modal pedestrian detection with misalignment based on modal-wise regression and multi-modal IoU

et al. 2023

Self Cite

View full text Add to dashboard Cite

Multi-modal pedestrian detection, which integrates visible and thermal sensors, has been developed to overcome many limitations of visible-modal pedestrian detection, such as poor illumination, cluttered background, and occlusion. By adopting the combination of multiple modalities, we can efficiently detect pedestrians even with poor visibility. Nevertheless, the critical assumption of multi-modal pedestrian detection is that multi-modal images are perfectly aligned. In general, however, this assumption often becomes invalid in real-world situations. Viewpoints of the different modal sensors are usually different. Then, the positions of pedestrians on the different modal images have disparities. We proposed a multi-modal faster-RCNN specifically designed to handle misalignment between two modalities. The faster-RCNN consists of a region proposal network (RPN) and a detector. We introduce position regressors for both modalities in the RPN and the detector. Intersection over union (IoU) is one of the useful metrics for object detection but is defined only for a single-modal image. We extend it into multi-modal IoU to evaluate the preciseness of both modalities. Our experimental results with the proposed evaluation metrics demonstrate that the proposed method has comparable performance with state-of-the-art methods and outperforms them for data with significant misalignment. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

show abstract