Multimodal Material Segmentation

Liang, Yupeng; Wakaki, Ryosuke; Nobuhara, Shohei; Nishino, Ko

doi:10.1109/cvpr52688.2022.01918

Cited by 21 publications

(35 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, these two results, and the intensity RGB image are fed into three independent conformer encoders 67 and fused using local and global guidance. In a more complex scenario, Liang et al 36 built a network to fuse RGB, infra-red, and polarization cues to produce an outdoor scene segmentation based on the object material type. The proposed pipeline is composed of two core elements: a network that will classify the objects present in one class of a subset of the segmentation labels from the CityScapes dataset 68 and a region-based filter selection module that chooses the modality that provides the most relevant information for determining the type of material of the constitutive elements of each detected object.…”

Section: Image Segmentationmentioning

confidence: 99%

“…Mei et al 37 introduced a medium-scale dataset, with 4511 images annotated only for the labels glass and no-glass. Similarly, Liang et al 36 made publicly available a dataset of semantic segmentation of urban scenes, with multi-modal sensors, but it only includes 500 labeled images, and Li et al 39 did it for road segmentation, and with their personalized infra-red polarization camera. Thus, there is a need for a common large-scale benchmark to evaluate the performance of these different segmentation algorithms to trace the direction toward a generalization of the polarization modality.…”

Section: Image Segmentationmentioning

confidence: 99%

“…38 (g)-(i) Input image, RGB-only method results, and the results of the multimodal material segmentation algorithm. 36 Images courtesy of the respective works.…”

Section: Surface Normal and Depth Estimationmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

Rodriguez,

Lew-Yan-Voon,

Martins

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Polarization information of the light can provide rich cues for computer vision and scene understanding tasks, such as the type of material, pose, and shape of the objects. With the advent of new and cheap polarimetric sensors, this imaging modality is becoming accessible to a wider public for solving problems, such as pose estimation, 3D reconstruction, underwater navigation, and depth estimation. However, we observe several limitations regarding the usage of this sensorial modality, as well as a lack of standards and publicly available tools to analyze polarization images. Furthermore, although polarization camera manufacturers usually provide acquisition tools to interface with their cameras, they rarely include processing algorithms that make use of the polarization information. In this work, we review recent advances in applications that involve polarization imaging, including a comprehensive survey of recent advances on polarization for vision and robotics perception tasks. We also introduce a complete software toolkit that provides common standards to communicate with and process information from most of the existing micro-grid polarization cameras on the market. The toolkit also implements several image processing algorithms for this modality, and it is publicly available on GitHub.

show abstract

Section: Image Segmentationmentioning

confidence: 99%

Section: Image Segmentationmentioning

confidence: 99%

“…38 (g)-(i) Input image, RGB-only method results, and the results of the multimodal material segmentation algorithm. 36 Images courtesy of the respective works.…”

Section: Surface Normal and Depth Estimationmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

Rodriguez,

Lew-Yan-Voon,

Martins

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Additional visual modalities include near-infrared images (Salamati et al, 2014;Liang et al, 2022), thermal images (Ha et al, 2017;Sun et al, 2019d), depth (Wang et al, 2015;Qi et al, 2017;Schneider et al, 2017), surfacenormals (Eigen and Fergus, 2015), 3D LiDAR point clouds (Kim et al, 2018;Jaritz et al, 2018;Caltagirone et al, 2019), etc. .…”

Section: Sis With Additional Modalitiesmentioning

confidence: 99%

Semantic Image Segmentation: Two Decades of Research

Volpi¹,

Chidlovskii²

2022

View full text Add to dashboard Cite

Semantic image segmentation (SiS) plays a fundamental role in a broad variety of computer vision applications, providing key information for the global understanding of an image. This survey is an effort to summarize two decades of research in the field of SiS, where we propose a literature review of solutions starting from early historical methods followed by an overview of more recent deep learning methods including the latest trend of using transformers. We complement the review by discussing particular cases of the weak supervision and side machine learning techniques that can be used to improve the semantic segmentation such as curriculum, incremental or self-supervised learning.State-of-the-art SiS models rely on a large amount of annotated samples, which are more expensive to obtain than labels for tasks such as image classification. Since unlabeled data is instead significantly cheaper to obtain, it is not surprising that Unsupervised Domain Adaptation (UDA) reached a broad success within the semantic segmentation community. Therefore, a second core contribution of this book is to summarize five years of a rapidly growing field, Domain Adaptation for Semantic Image Segmentation (DASiS) which embraces the importance of semantic segmentation itself and a critical need of adapting segmentation models to new environments. In addition to providing a comprehensive survey on DASiS techniques, we unveil also newer trends

show abstract