Multimodal Sequential Fashion Attribute Prediction

Abstract

We address multimodal product attribute prediction of fashion items based on product images and titles. The product attributes, such as type, sub-type, cut or fit, are in a chain format, with previous attribute values constraining the values of the next attributes. We propose to address this task with a sequential prediction model that can learn to capture the dependencies between the different attribute values in the chain. Our experiments on three product datasets show that the sequential model outperforms two non-sequential baselines on all experimental datasets. Compared to other models, the sequential model is also better able to generate sequences of attribute chains not seen during training. We also measure the contributions of both image and textual input and show that while text-only models always outperform image-only models, only the multimodal sequential model combining both image and text improves over the text-only model on all experimental datasets.

Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction

“…Visual Attributes. Visual attribute prediction (VAP) is a multi-label classification task that has been widely studied [42,4,2,22,38,23,32,30]. There are two main types of VAP tasks, as illustrated in Figure 1.…”

“…For example, they may only be related to low-level characteristics, such as color, which makes the task more interesting. In the fashion domain, some works [42,23,22,4] have used landmarks for clothing items, pose detection, or textual item descriptions to improve overall accuracy. However, such strong requirements regarding auxiliary information limit such methods to domain-specific solutions.…”

“…Domain-agnostic training. To achieve state-of-the-art performance on benchmarks, recent works [42,23,22,4] have relied on domain-specific auxiliary information during training. However, such methods have obvious limitations in terms of expansion to additional domains, which is an important feature for real applications.…”

“…2. Conceptual summary of previous methods and the proposed method denoted, as common [42,23,22,4], progressive [2], and MTSS. manner utilizing unlabeled images.…”

Prediction of sgRNA Off-Target Activity in CRISPR/Cas9 Gene Editing Using Graph Convolution Network

“…However, these traditional machine learning methods cannot take raw features from large, annotated datasets and use them to identify the patterns buried inside them. Deep learning algorithms is a powerful approach for learning complex patterns and has led to multiple performance breakthroughs in many research fields, including computer vision [ 32 , 33 , 34 , 35 , 36 , 37 ] and natural language processing [ 38 , 39 , 40 ]. However, very few prediction models have implemented the concept of deep learning into the sgRNA off-target propensity prediction problem.…”