Fashion-focused creative commons social dataset

Loni, Babak; Menéndez, María; Géorgescu, Mihai; Galli, Luca; Massari, Claudio; Altıngövde, İsmail Sengör; Martinenghi, Davide; Melenhorst, Mark; Vliegendhart, Raynor; Larson, Martha

doi:10.1145/2483977.2483984

Cited by 14 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The images were obtained from the Fashionfocused creative commons social dataset [17], which consists of user-generated content images related to fashion. On average, each image was used in 2.8 games (standard deviation 1.7), and each player participated in 12.8 games (standard deviation 39.4).…”

Section: Methodsmentioning

confidence: 99%

Robust aggregation of GWAP tracks for local image annotation

Bernaschina

Fraternali

Galli

et al. 2014

Proceedings of International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

The possibility of assigning labels to localized regions in an image enables flexible image retrieval paradigms. However, the process of automatically segmenting and tagging images is notoriously hard, due to the presence of occlusions, noise, challenging illumination conditions, background clutter, etc. For this reason, human computation has recently emerged as a viable alternative when computer vision algorithms fail to provide a satisfactory answer. For example, Games with a purpose (GWAP) represent a powerful crowdsourcing mechanism to collect implicit annotations from human players. In this paper we consider the problem of aggregating the gaming tracks collected by a GWAP we developed to solve challenging instances of image segmentation problems. In particular we consider the existence of malicious players, who might try to fool the rules of the game to achieve higher rewards. The proposed solution can automatically estimate the reliability of human players, thus identifying cheaters. This information is exploited to aggregate the gaming tracks, thus significantly improving the image segmentation result and the quality of local image annotations.

show abstract

Section: Methodsmentioning

confidence: 99%

Robust aggregation of GWAP tracks for local image annotation

Bernaschina

Fraternali

Galli

et al. 2014

Proceedings of International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…In this section, we introduce the six real crowdsourced data sets used in the experiments. The following is a detailed introduction of these six data sets: Fashion1 [51]: It is a binary crowdsourced data set to judge whether image data is related to fashion. The data set contains 4711 images.…”

Section: Real Datasetsmentioning

confidence: 99%

Robust Sparse Weighted Classification For Crowdsourcing

Hyeong-keun

Zhang

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Data collected from nature is usually unlabeled, and it is difficult to be used directly. This issue is well addressed by crowdsourcing, which provides a reasonable way for effectively using these unlabeled data. Generally, workers in crowdsourcing tasks are not professionals, so it is hard to obtain high-quality labels. To address this issue, a robust sparse weighted classification algorithm is proposed, which try to adjust the samples that are not correctly classified in the original lables as much as possible. Specifically, we evalute the ability of different workers(indicator weight matrix) to accurately label different samples by fitting the real data matrix to its weighted reconstruction matrix. And then, l 2,1 -norm and worker labeling ability similarity matrix are added, and negative effects of some bad workers are eliminated through the row sparsity property of l 2,1 -norm. Finally, the optimal indicator weight matrix is obtained by optimizing the two matrices in the objective function simultaneously. Therefore, the obtained optimal indicator weight matrix takes the similarity of worker labeling ability into consideration, and infers all the predicted labels. The results on synthetic and real data sets demonstrate that our algorithm is superior to other state-of-the-art methods.

show abstract

“…Intuitively speaking, it has multiple correct answers but at least one incorrect answer. For instance, consider the task to judge if an image contains a person wearing fashionable clothes or not [38,39]. Workers can disagree on what the correct answers are for an image that contains a clothes-wearing person as they have different preferences for fashion styles.…”

Section: Crowdsourcing Aspectsmentioning

confidence: 99%

A Technical Survey on Statistical Modelling and Design Methods for Crowdsourcing Quality Control

Jin

Carman

Zhu

et al. 2018

Preprint

View full text Add to dashboard Cite

Online crowdsourcing provides a scalable and inexpensive means to collect knowledge (e.g. labels) about various types of data items (e.g. text, audio, video). However, it is also known to result in large variance in the quality of recorded responses which often cannot be directly used for training machine learning systems. To resolve this issue, a lot of work has been conducted to control the response quality such that low-quality responses cannot adversely affect the performance of the machine learning systems. Such work is referred to as the quality control for crowdsourcing. Past quality control research can be divided into two major branches: quality control mechanism design and statistical models. The first branch focuses on designing measures, thresholds, interfaces and workflows for payment, gamification, question assignment and other mechanisms that influence workers' behaviour. The second branch focuses on developing statistical models to perform effective aggregation of responses to infer correct responses. The two branches are connected as statistical models (i) provide parameter estimates to support the measure and threshold calculation, and (ii) encode modelling assumptions used to derive (theoretical) performance guarantees for the mechanisms. There are surveys regarding each branch but they lack technical details about the other branch. Our survey is the first to bridge the two branches by providing technical details on how they work together under frameworks that systematically unify crowdsourcing aspects modelled by both of them to determine the response quality. We are also the first to provide taxonomies of quality control papers based on the proposed frameworks. Finally, we specify the current limitations and the corresponding future directions for the quality control research.

show abstract

Fashion-focused creative commons social dataset

Cited by 14 publications

References 18 publications

Robust aggregation of GWAP tracks for local image annotation

Robust aggregation of GWAP tracks for local image annotation

Robust Sparse Weighted Classification For Crowdsourcing

A Technical Survey on Statistical Modelling and Design Methods for Crowdsourcing Quality Control

Contact Info

Product

Resources

About