Learning Type-Aware Embeddings for Fashion Compatibility

Vasileva, Mariya I.; Plummer, Bryan A.; Dusad, Krishna; Rajpal, Shreya; Kumar, Ranjitha; Forsyth, David

doi:10.48550/arxiv.1803.09196

Cited by 3 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shih et al [22] propose to make multiple projection points of a query image. Vasileva et al [28] learn different metric spaces for different type combinations. While evaluating the outfit compatibility, these methods take the average of all pairwise compatibility as the output and neglect the relationship between the pairwise compatibility and the overall compatibility.…”

Section: Visual Compatibility Learningmentioning

confidence: 99%

“…This will leads to an improper situation e.g., if a shirt matches a trouser, the trouser then matches a shoe, the consequence is the shoe is forced to match the shirt. Therefore, we use the projected embedding with respect to different fashion type combinations to address the above problems, which refers to [3,28].…”

Section: Comparison With Projected Embeddingmentioning

confidence: 99%

“…Pooling [14] 88.35 ± 0.26 57.28 ± 0.31 Concatenation [27] 83.40 ± 0.48 52.91 ± 0.59 Self-Attention [32] 79.65 ± 0.68 48.60 ± 0.70 CSN [28] 84.90 ± 0.52 57.06 ± 1.70 BiLSTM [5] 74.44 ± 0.95 45.41 ± 0.40 BiLSTM+VSE [5] 74…”

Section: Comparison With Projected Embeddingmentioning

confidence: 99%

“…Many works used multimedia information to answer the questions like "Is this outfit compatible?" [14,27,28], "How to generate a compatible outfit?" [5,19].…”

Section: Introductionmentioning

confidence: 99%

“…To effectively implement the comparison module, we follow the ideas from the projected embedding [3] and type-aware embedding [28]. To leverage both image and text information of fashion items, we adopt visual semantic embedding [12] to learn a common representation between them.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Wang

Zhong

2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Existing works about fashion outfit compatibility focus on predicting the overall compatibility of a set of fashion items with their information from different modalities. However, there are few works explore how to explain the prediction, which limits the persuasiveness and effectiveness of the model. In this work, we propose an approach to not only predict but also diagnose the outfit compatibility. We introduce an end-to-end framework for this goal, which features for: (1) The overall compatibility is learned from all type-specified pairwise similarities between items, and the backpropagation gradients are used to diagnose the incompatible factors.(2) We leverage the hierarchy of CNN and compare the features at different layers to take into account the compatibilities of different aspects from the low level (such as color, texture) to the high level (such as style). To support the proposed method, we build a new type-specified outfit dataset named Polyvore-T based on Polyvore dataset. We compare our method with the prior state-of-the-art in two tasks: outfit compatibility prediction and fillin-the-blank. Experiments show that our approach has advantages in both prediction performance and diagnosis ability.

show abstract

Section: Visual Compatibility Learningmentioning

confidence: 99%

Section: Comparison With Projected Embeddingmentioning

confidence: 99%

Section: Comparison With Projected Embeddingmentioning

confidence: 99%

“…Many works used multimedia information to answer the questions like "Is this outfit compatible?" [14,27,28], "How to generate a compatible outfit?" [5,19].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Wang

Zhong

2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

Context-Aware Visual Compatibility Prediction

Cucurull

Taslakian

Vázquez

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

102

View full text Add to dashboard Cite

How do we determine whether two or more clothing items are compatible or visually appealing? Part of the answer lies in understanding of visual aesthetics, and is biased by personal preferences shaped by social attitudes, time, and place. In this work we propose a method that predicts compatibility between two items based on their visual features, as well as their context. We define context as the products that are known to be compatible with each of these item. Our model is in contrast to other metric learning approaches that rely on pairwise comparisons between item features alone. We address the compatibility prediction problem using a graph neural network that learns to generate product embeddings conditioned on their context. We present results for two prediction tasks (fill in the blank and outfit compatibility) tested on two fashion datasets Polyvore and Fashion-Gen, and on a subset of the Amazon dataset; we achieve state of the art results when using context information and show how test performance improves as more context is used.

show abstract

Learning Fashion Compatibility Across Apparel Categories for Outfit Recommendation

Polanía

Gupte

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

This paper addresses the problem of generating recommendations for completing the outfit given that a user is interested in a particular apparel item. The proposed method is based on a siamese network used for feature extraction followed by a fully-connected network used for learning a fashion compatibility metric. The embeddings generated by the siamese network are augmented with color histogram features motivated by the important role that color plays in determining fashion compatibility. The training of the network is formulated as a maximum a posteriori (MAP) problem where Laplacian distributions are assumed for the filters of the siamese network to promote sparsity and matrix-variate normal distributions are assumed for the weights of the metric network to efficiently exploit correlations between the input units of each fully-connected layer.

show abstract

Learning Type-Aware Embeddings for Fashion Compatibility

Cited by 3 publications

References 0 publications

Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network

Context-Aware Visual Compatibility Prediction

Learning Fashion Compatibility Across Apparel Categories for Outfit Recommendation

Contact Info

Product

Resources

About