Diversity within the crowd

Kandasamy, Durga M.; Curtis, Kristal; Fox, Armando; Patterson, David A.

doi:10.1145/2141512.2141556

Cited by 7 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rashtchian and Young (2010) compared the quality of captions provided by paid contributors and expert annotators. Other work examined the potential benefits of combining expert and non-expert responses to achieve results of high quality with fewer overall responses (Kandasamy et al, 2012). This line of work informs the choice between high-quality yet costly expert contributions on the one hand, and somewhat lower quality but less expensive crowd contributions on the other hand.…”

Section: Related Workmentioning

confidence: 99%

To Play or Not to Play: Interactions between Response Quality and Task Complexity in Games and Paid Crowdsourcing

Krause

Kizilcec

2015

HCOMP

View full text Add to dashboard Cite

Digital games are a viable alternative to accomplish crowdsourcing tasks that would traditionally require paid online labor. This study compares the quality of crowdsourcing with games and paid crowdsourcing for simple and complex annotation tasks in a controlled exper-iment. While no difference in quality was found for the simple task, paid contributors’ response quality was sub-stantially lower than players’ quality for the complex task (92% vs. 78% average accuracy). Results suggest that crowdsourcing with games provides similar and potentially even higher response quality relative to paid crowdsourcing.

show abstract

Section: Related Workmentioning

confidence: 99%

To Play or Not to Play: Interactions between Response Quality and Task Complexity in Games and Paid Crowdsourcing

Krause

Kizilcec

2015

HCOMP

View full text Add to dashboard Cite

show abstract

“…[Heimerl et al, 2012] show that students can grade exams with 2% higher accuracy (at the same price) or at 33% lower cost (at equivalent accuracy) than traditional singleexpert grading. [Kandasamy et al, 2012] explore how expert input can be combined with input from members of the general population. This provides a motivation for attendeesourcing.…”

Section: Related Workmentioning

confidence: 99%

Attendee-Sourcing: Exploring The Design Space of Community-Informed Conference Scheduling

Bhardwaj¹,

Kim²,

Dow

et al. 2014

HCOMP

View full text Add to dashboard Cite

Constructing a good conference schedule for a large multi-track conference needs to take into account the preferences and constraints of organizers, authors, and attendees. Creating a schedule which has fewer conflicts for authors and attendees, and thematically coherent sessions is a challenging task. Cobi introduced an alternative approach to conference scheduling by engaging the community to play an active role in the planning process. The current Cobi pipeline consists of committee-sourcing and author-sourcing to plan a conference schedule. We further explore the design space of community-sourcing by introducing attendee-sourcing -- a process that collects input from conference attendees and encodes them as preferences and constraints for creating sessions and schedule. For CHI 2014, a large multi-track conference in human-computer interaction with more than 3,000 attendees and 1,000 authors, we collected attendees’ preferences by making available all the accepted papers at the conference on a paper recommendation tool we built called Confer, for a period of 45 days before announcing the conference program (sessions and schedule). We compare the preferences marked on Confer with the preferences collected from Cobi’s author-sourcing approach. We show that attendee-sourcing can provide insights beyond what can be discovered by author-sourcing. For CHI 2014, the results show value in the method and attendees’ participation. It produces data that provides more alternatives in scheduling and complements data collected from other methods for creating coherent sessions and reducing conflicts.

show abstract