Fine-grained Visual Faceted Search

International audienceLarge-scale annotated corpora have yielded impressive performance improvements in computer vision and multimedia content analysis. However, such datasets depend on an enormous amount of human labeling effort. When the labels correspond to well-known concepts, it is straightforward to train the annotators by giving a few examples with known answers. It is also straightforward to judge the quality of their labels. Neither is true when there are thousands of complex domain-specific labels. Training on all labels is infeasible and the quality of an annotator's judgements may be vastly different for some subsets of labels than for others. This paper proposes a set of data-driven algorithms to 1) train image annotators on how to disambiguate among automatically generated candidate labels, 2) evaluate the quality of annotators' label suggestions, and 3) weigh predictions. The algorithms adapt to the skills of each annotator both in the questions asked and the weights given to their answers. The underlying judgements are Bayesian, based on adaptive priors. We measure the benefits of these algorithms on a live user experiment related to image-based plant identification involving around 1000 people. The proposed methods are shown to enable huge gains in annotation accuracy. A standard user can correctly label around 2% of our data. This goes up to 80% with machine learning assisted training and assignment and up to almost 90% when doing a weighted combination of several annotators' labels

show abstract

Crowdsourcing Thousands of Specialized Labels: A Bayesian Active Training Approach

Servajean

Joly

Shasha

et al. 2017

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Fine-grained Visual Faceted Search

Abstract: International audienc

Cited by 1 publication

References 8 publications

Crowdsourcing Thousands of Specialized Labels: A Bayesian Active Training Approach

Crowdsourcing Thousands of Specialized Labels: A Bayesian Active Training Approach

Contact Info

Product

Resources

About