Sampling Matters in Deep Embedding Learning

Manmatha, R.; Wu, Chao-Yuan; Smola, Alexander J.; Krähenbühl, Philipp

doi:10.1109/iccv.2017.309

Cited by 622 publications

(424 citation statements)

References 33 publications

Supporting

Mentioning

420

Contrasting

Unclassified

Order By: Relevance

“…proposed SGML framework is compared with state-of-theart methods using various sampling methods [26], [61], loss functions [41] [61] and ensemble methods [1], [42], [53]. Our SGML achieves a Recall@1 of 71.9% outperforming methods such as distance-weighted [26] and position-dependent sampling [61], and loss functions such as pair-wise loss [26], triplet/quadratic loss [61] and binomial deviance loss [41]. This clearly shows the advantage of the proposed SBDL loss and its inherent property of treating training samples based on their degree of information.…”

Section: Results: Deepfashion Dataset 1) Comparison With State-of-mentioning

confidence: 99%

Dynamically Modulated Deep Metric Learning for Visual Search

Manandhar

Baştan

Yap

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep metric learning applied to various multimedia applications has shown promising results in tasks such identification, retrieval and recognition. Existing metric learning methods often do not consider different granularity in visual similarity. However, in many domain applications, images exhibit similarity at multiple granularities with visual semantic concepts, e.g. fashion demonstrates similarity ranging from clothing of the exact same instance to similar looks/design or a common category. Therefore, training image triplets/pairs used for metric learning inherently possess different degree of information. However, the existing methods often treats them with equal importance during training. This hinders capturing the underlying granularities in feature similarity which is required for effective visual search.In view of this, we propose a new deep semantic granularity metric learning (SGML) that develops a novel idea of detecting and leveraging attribute semantic space to capture different granularity of similarity, and then integrate this information into deep metric learning. The proposed framework simultaneously learns image attributes and embeddings using multitask CNNs with shared parameters. The two tasks are not only jointly optimized but are further linked by the semantic granularity similarity mappings to leverage the correlations between the tasks. To this end, we propose a new soft-binomial deviance loss that effectively integrates the degree of information in training samples, which helps to capture visual similarity at multiple granularities. Compared to recent ensemble-based methods, our framework is conceptually elegant, computationally simple and provides better performance. We perform extensive experiments on benchmark metric learning datasets and demonstrate that our method outperforms recent state-of-the-art methods, e.g. , 1-4.5% improvement in Recall@1 over the previous state-of-the-arts [1], [2] on DeepFashion In-Shop dataset.

show abstract

Section: Results: Deepfashion Dataset 1) Comparison With State-of-mentioning

confidence: 99%

Dynamically Modulated Deep Metric Learning for Visual Search

Manandhar

Baştan

Yap

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…We set the dimensionality of φ k to D = 2048 and choose the parameters λ 1 , λ 2 , Σ using crossvalidation on the training set. The margin parameter γ is set to 0.2 as suggested in various works [33]. We train our model using stochastic gradient descent using an initial learning rate of 0.01 and momentum of 0.9.…”

Section: Implementation Details and Benchmarkingmentioning

confidence: 99%

“…Full model vs. triplet learning:. In this ablation experiment (Tab.3 (Right) Triplets Only) we use our extracted reliable (dis-)similarity relationships to mine triplet-constraints as input into a standard triplet-loss framework [33]. For a sample that serves as triplet anchor, reliable similarity relations act as positive constraints and reliable dissimilarity relations act as negative constraints.…”

Section: Ablation Studiesmentioning

confidence: 99%

Unsupervised representation learning by discovering reliable image relations

Milbich

Ghori

Diego

et al. 2020

Pattern Recognition

View full text Add to dashboard Cite

Learning robust representations that allow to reliably establish relations between images is of paramount importance for virtually all of computer vision. Annotating the quadratic number of pairwise relations between training images is simply not feasible, while unsupervised inference is prone to noise, thus leaving the vast majority of these relations to be unreliable. To nevertheless find those relations which can be reliably utilized for learning, we follow a divide-and-conquer strategy: We find reliable similarities by extracting compact groups of images and reliable dissimilarities by partitioning these groups into subsets, converting the complicated overall problem into few reliable local subproblems. For each of the subsets we obtain a representation by learning a mapping to a target feature space so that their reliable relations are kept. Transitivity relations between the subsets are then exploited to consolidate the local solutions into a concerted global representation. While iterating between grouping, partitioning, and learning, we can successively use more and more reliable relations which, in turn, improves our image representation. In experiments, our approach shows state-of-the-art performance on unsupervised classification on ImageNet with 46.0% and competes favorably on different transfer learning tasks on PASCAL VOC.

show abstract

“…Within this framework, techniques to sample triplets remains an active area of research. These include techniques such as hard-negative mining [7], semi-hard negative mining [21] and distance weighted sampling [28] to bias the sampling of triplets.…”

Section: Related Workmentioning

confidence: 99%

Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation

Sharma

Kalogerakis

Maji

2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

User generated 3D shapes in online repositories contain rich information about surfaces, primitives, and their geometric relations, often arranged in a hierarchy. We present a framework for learning representations of 3D shapes that reflect the information present in this meta data and show that it leads to improved generalization for semantic segmentation tasks. Our approach is a point embedding network that generates a vectorial representation of the 3D points such that it reflects the grouping hierarchy and tag data. The main challenge is that the data is noisy and highly variable. To this end, we present a tree-aware metriclearning approach and demonstrate that such learned embeddings offer excellent transfer to semantic segmentation tasks, especially when training data is limited. Our approach reduces the relative error by 10.2% with 8 training examples, by 11.72% with 120 training examples on the ShapeNet semantic segmentation benchmark, in comparison to the network trained from scratch. By utilizing tag data the relative error is reduced by 12.8% with 8 training examples, in comparison to the network trained from scratch. These improvements come at no additional labeling cost as the meta data is freely available.

show abstract

Sampling Matters in Deep Embedding Learning

Cited by 622 publications

References 33 publications

Dynamically Modulated Deep Metric Learning for Visual Search

Dynamically Modulated Deep Metric Learning for Visual Search

Unsupervised representation learning by discovering reliable image relations

Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation

Contact Info

Product

Resources

About