Neural Congealing: Aligning Images to a Joint Semantic Atlas

Abstract

Project webpage: https://neural-congealing.github.io/ Input Images Congealed Images Edited ImagesFigure 1. Given a set of input images, our method automatically detects and jointly aligns semantically-common content across the images. This is achieved through a test-time training approach that estimates a unified 2D atlas that represents the common semantic content, and dense mappings from the joint atlas to each of the input images. Our atlas and mappings are optimized per input set in a self-supervised manner by leveraging a pre-trained DINO-ViT model. Our method can be applied to diverse image sets, without requiring any additional training data, and allows us to automatically propagate an edit applied to a single image across the entire set.

Learning Universal Semantic Correspondences with No Supervision and Automatic Data Curation

“…It achieves good performance, but it is limited to classes where a pre-trained GAN is available. Similarly, [40] learn a 2D atlas for a given class, but their method only works for a limited number of classes and needs careful manual curation of the images used. [68] learn semantic correspondences using 3D cycle consistency, requiring 3D CAD models during training.…”