Learning Fashion Compatibility Across Apparel Categories for Outfit Recommendation

Abstract

This paper addresses the problem of generating recommendations for completing the outfit given that a user is interested in a particular apparel item. The proposed method is based on a siamese network used for feature extraction followed by a fully-connected network used for learning a fashion compatibility metric. The embeddings generated by the siamese network are augmented with color histogram features motivated by the important role that color plays in determining fashion compatibility. The training of the network is formulated as a maximum a posteriori (MAP) problem where Laplacian distributions are assumed for the filters of the siamese network to promote sparsity and matrix-variate normal distributions are assumed for the weights of the metric network to efficiently exploit correlations between the input units of each fully-connected layer.

A Review of Modern Fashion Recommender Systems

“…Depending on the task, fashion item and outfit recommendation, and size recommendation as we identified in Section 2.1, a variety of other models have been used in the studied research work. For example, RNNs [21,92], graph-modeling [27,87,107], two-input (Siamese) CNNs [79,113,158,164,167], using attention mechanisms [92] and so forth. Detailed discussion on these approaches is left as a future direction.…”

“…where 𝑠 is the outfit utility function, a scoring function that takes into account different types of relationships between fashion items to generate an outfit compatibility score. It is worth noting that, given this general task definition, the evaluation of fashion outfit composition is typically performed via fill-in-the-blank (FITB) or outfit compatibility score prediction [113] described in Section 4. □ Moreover, it is possible to encode several objectives relevant to the fashion domain in the definition of the outfit composition scoring function by incorporating domain knowledge.…”

“…A number of works have tried to model visual/style compatibility as a metric learning problem, for instance [54,107,113] for clothes and [6,16] for furniture. Yin et al [158] propose a fashion recommendation system that, in the first phase, learns and incorporates visual compatibility relationships of items at the pixel level.…”

“…In the second phase, a domain adaptation strategy is used to allow the use of the knowledge extracted from the source domain that can be consistent with a different domain, that is, how to make sure that the knowledge learned in the database used in their experiments can also be compatible with others. Polanía and Gupte [113] proposed a neural model composed of two sub-networks, first a Siamese sub-network which extracts visual features from a pair of images and a metric learning approach that maps the pair of features and auxiliary cues (color information) to a fashion compatibility score. Li et al [88] proposed learning a representation of an outfit by exploiting the information from the image, title, and category and a multi-modal fusion model.…”

Fashion Recommendation Systems, Models and Methods: A Review

“…Fashion recommendation systems (FRSs) generally provide specific recommendations to the consumer based on their browsing and previous purchase history. Social-network-based FRSs consider the user's social circle, fashion product attributes, image parsing, fashion trends, and consistency in fashion styles as important factors since they impact upon the user's purchasing decisions [28][29][30][31][32][33][34][35][36][37][38]. FRSs have the ability to reduce transaction costs for consumers and increase revenue for retailers.…”