2014
DOI: 10.1109/jstsp.2014.2316116
|View full text |Cite
|
Sign up to set email alerts
|

Reliable Crowdsourcing for Multi-Class Labeling Using Coding Theory

Abstract: Crowdsourcing systems often have crowd workers that perform unreliable work on the task they are assigned. In this paper, we propose the use of error-control codes and decoding algorithms to design crowdsourcing systems for reliable classification despite unreliable crowd workers. Coding-theory based techniques also allow us to pose easyto-answer binary questions to the crowd workers. We consider three different crowdsourcing models: systems with independent crowd workers, systems with peer-dependent reward sc… Show more

Help me understand this report
View preprint versions

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

0
55
0

Year Published

2014
2014
2021
2021

Publication Types

Select...
5
3
1

Relationship

1
8

Authors

Journals

citations
Cited by 58 publications
(55 citation statements)
references
References 20 publications
0
55
0
Order By: Relevance
“…In our research we will focus on this approach owing to its higher accuracy than previously described approaches [8,12].…”
Section: Multi-classifier By Hamming-distance Decisionmentioning
confidence: 99%
“…In our research we will focus on this approach owing to its higher accuracy than previously described approaches [8,12].…”
Section: Multi-classifier By Hamming-distance Decisionmentioning
confidence: 99%
“…When team members are rational and knowledgeable about the given problem domain, the team decision theory provides well-developed insights [14]. Since the emergence of high-performance computing and networking infrastructure, new paradigms for user participation, such as crowdsourcing, have arisen for distributed inference tasks [14]. Therefore, crowdsourcing can be defined as a distributed approach for complex problems through an open user participation paradigm.…”
Section: Crowdsourcingmentioning
confidence: 99%
“…The best phoneme transcription is then the transcription whose error-correcting code, c mj , best matches the distinctive feature labels that were actually provided by the labelers. Redundancy in this way permits us to acquire more accurate transcriptions, because even a crowd worker who is wrong about every single phoneme is often, nevertheless, right about many of the distinctive features (Vempaty et al 2014). …”
Section: Crowdsourcing With Binary Error Correcting Codesmentioning
confidence: 99%