Jointly Aligning Millions of Images With Deep Penalised Reconstruction Congealing

Annunziata, Roberto; Sagonas, Christos; Calì, Jacques

doi:10.1109/iccv.2019.00017

Cited by 9 publications

(8 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 2 illustrates the STN for classification task, where X o is the original input image, E 1 is the spatial transformer block, M is the estimated alignment matrix, Xb is the aligned image, and E 2 is the classification network. The loss function is L = min ||y − ŷ|| 2 2 , where y and ŷ are the ground truth and predicted image class label.…”

Section: Background: Spatial Transformer Networkmentioning

confidence: 99%

“…In adaptive Gabor convolutional networks [20], the convolutional kernels are adaptively multiplied by Gabor filters to achieve invariant information extracted from VOLUME 4, 2016 images. The spatial transformer network (STN) [4] is used in [2] to tackle the joint image alignment problem on larger datasets with higher variability. STN [4] can spatially transform the input images by embedding the spatial transformer block into a target network or algorithm.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Recurrent Affine Transform Encoder for Image Representation

et al. 2022

View full text Add to dashboard Cite

This paper proposes a Recurrent Affine Transform Encoder (RATE) that can be used for image representation learning. We propose a learning architecture that enables a CNN encoder to learn the affine transform parameter of images. The proposed learning architecture decomposes an affine transform matrix into two transform matrices and learns them jointly in a self-supervised manner. The proposed RATE is trained by unlabeled image data without any ground truth and infers the affine transform parameter of input images recurrently. The inferred affine transform parameter can be used to represent images in canonical form to greatly reduce the image variations in affine transforms such as rotation, scaling, and translation. Different from the spatial transformer network, the proposed RATE does not need to be embedded into other networks for training with the aid of other learning objectives. We show that the proposed RATE learns the affine transform parameter of images and achieves impressive image representation results in terms of invariance to translation, scaling, and rotation. We also show that the classification performance is enhanced and is more robust against distortion by incorporating the RATE into the existing classification model.INDEX TERMS Canonical image base, self-supervised learning, representation learning.

show abstract

Section: Background: Spatial Transformer Networkmentioning

confidence: 99%

mentioning

confidence: 99%

Recurrent Affine Transform Encoder for Image Representation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Several works [54,42,11,12] developed a similar idea for continuous parametric transformations in the simpler setting of image alignment, which was later applied again for clustering by [48,53,45,2]. Recently, Monnier et al [55] generalize these ideas to global alignments and large-scale datasets by leveraging neural networks to predict spatial alignments -implemented as spatial transformers [29] -, color transformations and morphological modifications.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Layered Image Decomposition into Object Prototypes

Monnier,

Vincent,

Ponce

et al. 2021

Preprint

View full text Add to dashboard Cite

We present an unsupervised learning framework for decomposing images into layers of automatically discovered object models. Contrary to recent approaches that model image layers with autoencoder networks, we represent them as explicit transformations of a small set of prototypical images. Our model has three main components: (i) a set of object prototypes in the form of learnable images with a transparency channel, which we refer to as sprites; (ii) differentiable parametric functions predicting occlusions and transformation parameters necessary to instantiate the sprites in a given image; (iii) a layered image formation model with occlusion for compositing these instances into complete images including background. By jointly learning the sprites and occlusion/transformation predictors to reconstruct images, our approach not only yields accurate layered image decompositions, but also identifies object categories and instance parameters. We first validate our approach by providing results on par with the state of the art on standard multiobject synthetic benchmarks (Tetrominoes, Multi-dSprites, CLEVR6). We then demonstrate the applicability of our model to real images in tasks that include clustering (SVHN, GTSRB), cosegmentation (Weizmann Horse) and object discovery from unfiltered social network images. To the best of our knowledge, our approach is the first layered image decomposition algorithm that learns an explicit and shared concept of object type, and is robust enough to be applied to real images.

show abstract

“…Deep learning was recently used to scale the idea of congealing for global alignment of a single class of images [32] or time series [33]. Both works build on the idea of Spatial Transformer Networks [34] (STN) that spatial transformation are differentiable and can be learned by deep networks.…”

Section: Related Workmentioning

confidence: 99%

Deep Transformation-Invariant Clustering

Monnier,

Groueix,

Aubry

2020

Preprint

View full text Add to dashboard Cite

Recent advances in image clustering typically focus on learning better deep representations. In contrast, we present an orthogonal approach that does not rely on abstract features but instead learns to predict transformations and performs clustering directly in pixel space. This learning process naturally fits in the gradient-based training of K-means and Gaussian mixture model, without requiring any additional loss or hyper-parameters. It leads us to two new deep transformation-invariant clustering frameworks, which jointly learn prototypes and transformations. More specifically, we use deep learning modules that enable us to resolve invariance to spatial, color and morphological transformations. Our approach is conceptually simple and comes with several advantages, including the possibility to easily adapt the desired invariance to the task and a strong interpretability of both cluster centers and assignments to clusters. We demonstrate that our novel approach yields competitive and highly promising results on standard image clustering benchmarks. Finally, we showcase its robustness and the advantages of its improved interpretability by visualizing clustering results over real photograph collections.Preprint. Under review.

show abstract

Jointly Aligning Millions of Images With Deep Penalised Reconstruction Congealing

Cited by 9 publications

References 44 publications

Recurrent Affine Transform Encoder for Image Representation

Recurrent Affine Transform Encoder for Image Representation

Unsupervised Layered Image Decomposition into Object Prototypes

Deep Transformation-Invariant Clustering

Contact Info

Product

Resources

About