Motion-supervised Co-Part Segmentation

Siarohin, Aliaksandr; Roy, Subhankar; Lathuilière, Stéphane; Tulyakov, Sergey; Ricci, Elisa; Sebe, Nicu

doi:10.1109/icpr48806.2021.9412520

Cited by 16 publications

(22 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following related work [15,30,46], we analyze the improvement of our method in terms of the positioning of parts and the mask coverage on the established benchmarks. We provide results for a wide variety of images containing faces, animals, flowers, and humans.…”

Section: Methodsmentioning

confidence: 99%

“…For example, in portrait images, the hair is often not masked. Temporal information can be used [9,46] to achieve better segmentation results. In comparison to all of these approaches, our model uses less information (single images without video or saliency map) yet outperform these on half of the most established metrics and datasets, as we evaluate in our experiments, Section 4.…”

Section: Related Workmentioning

confidence: 99%

“…This paper tackles the problem of unsupervised part segmentation. Discovering object parts in images is a fundamental problem in computer vision as parts provide an intermediate representation that is robust to object appearance and pose variations [15,46]. Many high-level tasks benefit from part representations, such as 3D reconstruction [29,63], pose estimation [25,40], and image editing [13,62].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

He¹,

Wandt²,

Rhodin³

2021

Preprint

View full text Add to dashboard Cite

Segmenting an image into its parts is a common preprocess for high-level vision tasks such as image editing. However, annotating masks for supervised training is expensive. Weakly-supervised and unsupervised methods exist, but they depend on the comparison of pairs of images, such as from multi-views, frames of videos, and image transformations of single images, which limit their applicability. To address this, we propose a GAN-based approach that generates images conditioned on latent masks, thereby alleviating full or weak annotations required by previous approaches. We show that such mask-conditioned image generation can be learned faithfully when conditioning the masks in a hierarchical manner on 2D latent points that define the position of parts explicitly. Without requiring supervision of masks or points, this strategy increases robustness of mask to viewpoint and object position changes. It also lets us generate image-mask pairs for training a segmentation network, which outperforms state-of-the-art unsupervised segmentation methods on established benchmarks.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

He¹,

Wandt²,

Rhodin³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These works significantly demonstrate that motion information can and should be adopted for inferring meaningful object parts. Motion-supervised co-part segmentation [30], the pioneering work proposed a novel architecture via an approach of self-supervised and reconstruction for co-part segmentation. It constitutes intermediate motion representations robust to sensor changes and appearance variations, as well as eliminates the background disturbance.…”

Section: Motion Descriptionmentioning

confidence: 99%

“…The motion flow estimator aims to predict a dense motion field from a referred body image to a target body image with the same identity. Inspired by previous approaches [29,30], the motion estimator module proceeds in two steps. Firstly, we approximate both transformations from sets of sparse trajectories, obtained by using keypoints learned in a self-supervised manner.…”

Section: Motion Flow Estimatormentioning

confidence: 99%

Motion-Aware Transformer For Occluded Person Re-identification

Zhou¹,

Liu²,

Lv³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently, occluded person re-identification(Re-ID) remains a challenging task that people are frequently obscured by other people or obstacles, especially in a crowd massing situation. In this paper, we propose a selfsupervised deep learning method to improve the location performance for human parts through occluded person Re-ID. Unlike previous works, we find that motion information derived from the photos of various human postures can help identify major human body components. Firstly, a motion-aware transformer encoder-decoder architecture is designed to obtain keypoints heatmaps and part-segmentation maps. Secondly, an affine transformation module is utilized to acquire motion information from the keypoint detection branch. Then the motion information will support the segmentation branch to achieve refined human part segmentation maps, and effectively divide the human body into reasonable groups. Finally, several cases demonstrate the efficiency of the proposed model in distinguishing different representative parts of the human body, which can avoid the background and occlusion disturbs. Our method consistently achieves state-of-the-art results on several popular datasets, including occluded, partial, and holistic.

show abstract

Deepfake attribution: On the source identification of artificially generated images

Khoo

Phan

Lim

2021

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Synthetic media or "deepfakes" are making great advances in visual quality, diversity, and verisimilitude, empowered by large-scale publicly accessible datasets and rapid technical progress in deep generative modeling. Heralding a paradigm shift in how online content is trusted, researchers in digital image forensics have responded with different proposals to reliably detect AI-generated images in the wild. However, binary classification of image authenticity is insufficient to regulate the ethical usage of deepfake technology as new applications are developed. This article provides an overview of the major innovations in synthetic forgery detection as of 2020, while highlighting the recent shift in research towards ways to attribute AI-generated images to their generative sources with evidence. We define the various categories of deepfakes in existence, the subtle processing traces and fingerprints that distinguish AI-generated images from reality and each other, and the different degrees of attribution possible with current understanding of generative algorithms.Additionally, we describe the limitations of synthetic image recognition methods in practice, the counter-forensic attacks devised to exploit these limitations, and directions for new research to assure the long-term relevance of deepfake forensics. Reliable, explainable, and generalizable attribution methods would hold malicious users accountable for AI-enabled disinformation, grant plausible deniability to appropriate users, and facilitate intellectual property protection of deepfake technology.

show abstract

Motion-supervised Co-Part Segmentation

Cited by 16 publications

References 28 publications

GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation

Motion-Aware Transformer For Occluded Person Re-identification

Deepfake attribution: On the source identification of artificially generated images

Contact Info

Product

Resources

About