FisheyePixPro: Self-supervised pretraining using Fisheye images for semantic segmentation

Ramchandra, Cheke; Sistu, Ganesh; Eising, Ciaran; Pepijn, van de Ven; Varun, Ravi Kumar; Senthil, Yogamani

doi:10.2352/ei.2022.34.16.avm-147

Cited by 7 publications

(4 citation statements)

References 31 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditional approaches like [25], [26], [27] all choose positive instances by augmenting an image through some transformation and treating all other instances in the batch as the negative set. Within the domain of fisheye, [28] has utilized existing contrastive learning approaches on fisheye data for the task of semantic segmentation. This work differs fundamentally from ours in the sense that we are proposing a contrastive learning approach specifically geared towards creating a fisheye specific representation space, rather than a generic space based on previous learning approaches.…”

Section: Contrastive Learningmentioning

confidence: 99%

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Kiran

Prabhushankar

Yavuz

et al. 2023

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality. Fisheye data has the wider field of view advantage over other types of cameras, but this comes at the expense of high radial distortion. As a result, objects further from the center exhibit deformations that make it difficult for a model to identify their semantic context. While previous work has attempted architectural and training augmentation changes to alleviate this effect, no work has attempted to guide the model towards learning a representation space that reflects this interaction between distortion and semantic context inherent to fisheye data. We introduce an approach to exploit this relationship by first extracting distortion class labels based on an object's distance from the center of the image. We then shape a backbone's representation space with a weighted contrastive loss that constrains objects of the same semantic class and distortion class to be close to each other within a lower dimensional embedding space. This backbone trained with both semantic and distortion information is then fine-tuned within an object detection setting to empirically evaluate the quality of the learnt representation. We show this method leads to performance improvements by as much as 1.1% mean average precision over standard object detection strategies and .6% improvement over other state of the art representation learning approaches.

show abstract

Section: Contrastive Learningmentioning

confidence: 99%

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Kiran

Prabhushankar

Yavuz

et al. 2023

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

show abstract

“…DenseCL [15] implements self-supervision by optimizing a pairwise contrastive (dis)similarity loss between two views of input images, whereas pixel-level pretext tasks are introduced for learning dense feature representations in [72]. FisheyePix-Pro [73] attempts to pretrain a contrastive learning based model directly on fisheye images. Cross-image pixel contrast has been leveraged for semantic segmentation by looking beyond single images [74], [75], [76] and enforcing pixel embeddings belonging to the same semantic class to be more similar than embeddings from different classes.…”

Section: Unsupervised Dense Contrastive Learningmentioning

confidence: 99%

Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

Jaus

Yang

Stiefelhagen

2021

2021 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

In this work, we introduce panoramic panoptic segmentation as the most holistic scene understanding both in terms of field of view and image level understanding. A complete surrounding understanding provides a maximum of information to the agent, which is essential for any intelligent vehicle in order to make informed decisions in a safetycritical dynamic environment such as real-world traffic. In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain. Using our proposed method, we manage to achieve significant improvements of over 5% measured in PQ over nonadapted models on our Wild Panoramic Panoptic Segmentation (WildPPS) dataset. We show that our proposed Panoramic Robust Feature (PRF) framework is not only suitable to improve performance on panoramic images but can be beneficial whenever model training and deployment are executed on data taken from different distributions. As an additional contribution, we publish WildPPS: The first panoramic panoptic image dataset to foster progress in surrounding perception.

show abstract

“…Numerous pretext task techniques have been suggested to advance the field of self-supervised learning in the domains of natural images [15,16,17,18,19,20] and self-supervised learning methods have been used in a wide range of applications [21,22,23,24,25,26,27]. A common theme in the majority of these initiatives is that the use of the pretext results is limited to use on downstream tasks using the same dataset.…”

Section: Introductionmentioning

confidence: 99%

PatchLoc: Embedded Patch Localization Pretext Task for Tumor Segmentation in Medical Images

Cheke,

Eising,

Denny

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

Supervised deep learning methods have produced state-of-the-art results with large labeled datasets. However, accessing large labeled datasets is difficult in medical image analysis because of a shortage of medical experts, expensive annotations, and privacy constraints in the healthcare domain. Selfsupervised learning is a branch of machine learning that exploits unlabeled data to encourage network weights toward a valid latent representation of the data during a so-called pretext task. The features learned by the model while solving pretext tasks are transferred to a downstream task where limited annotations are available. In this work, we propose PatchLoc, a novel pretext task whose objective is to find the location of a given patch from an image as a source of supervision. We validated the effectiveness of PatchLoc on a downstream segmentation task using three different medical datasets. PatchLoc yields substantial improvements compared to U-Net trained from scratch and other pretext task-based approaches in a low data regime.INDEX TERMS Medical imaging, pretext tasks, self-supervised learning, limited annotations.

show abstract

FisheyePixPro: Self-supervised pretraining using Fisheye images for semantic segmentation

Cited by 7 publications

References 31 publications

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

PatchLoc: Embedded Patch Localization Pretext Task for Tumor Segmentation in Medical Images

Contact Info

Product

Resources

About