Curiosity Killed the Cat, but Makes Crowdwork Better

Abstract

Crowdsourcing systems are designed to elicit help from humans to accomplish tasks that are still difficult for computers. How to motivate workers to stay longer and/or perform better in crowdsourcing systems is a critical question for designers. Previous work have explored different motivational frameworks, both extrinsic and intrinsic. In this work, we examine the potential for curiosity as a new type of intrinsic motivational driver to incentivize crowd workers. We design crowdsourcing task interfaces that explicitly incorporate mechanisms to induce curiosity and conduct a set of experiments on Amazon's Mechanical Turk. Our experiment results show that curiosity interventions improve worker retention without degrading performance, and the magnitude of the effects are influenced by both the personal characteristics of the worker and the nature of the task.

Quality Control in Crowdsourcing

“…Krause and Kizilcec [2015] found that for complex tasks workers produce better results in a gamified than in a paid condition; for simple tasks there was no difference. Designing tasks that induce curiosity is a task-agnostic strategy to improve worker retention [Law et al 2016]. Feyisetan et al [2015] propose a model to predict optimal combinations of financial and gamified incentives.…”

Learning to Listen: Critically Considering the Role of AI in Human Storytelling and Character Creation

“…There are a number of approaches that already use crowd-powered "minicorpora" to teach AI how to generate narrative, (Guzdial et al, 2015;Li et al, 2012Li et al, , 2013Li et al, , 2014Purdy and Riedl, 2016) but this existing work does not seek to improve experiences or engage crowdworkers in meaningful forms of storytelling. Taking cues from work in improving crowd workers experiences by inducing curiosity (Law et al, 2016), we might further consider crowd-powered feedback mechanisms could allow both story creators and crowd workers to engage in and benefit from stories in different ways. For example, taking a character-centric approach, AI systems could prove useful in identifying when characters begin to fall into traps of stereotypes or implausibility.…”

Enhancing Crowdworkers' Vigilance

“…The crowdsourcing model encapsulates a key question that has captured the attention of researchers for a long time-How to motivate workers so they perform higher quality work. This question is of great importance especially when there is evidence for a group of workers who are primarily interested in producing quick labor rather than quality [Laws et al, 2011;Akkaya et al, 2010]. Various methods have been proposed to address this problem.…”

“…Various methods have been proposed to address this problem. Some works examine the use of different financial compensation mechanisms which are based on the workers' performance [Yin and Chen, 2015;Mason and Watts, 2010;Gao et al, 2012;Feng et al, 2014]. Another line of work [Law et al, 2016;Kaufmann et al, 2011] suggests the use of intrinsic behavioral factors to motivate workers.…”

“…Some works examine the use of different financial compensation mechanisms which are based on the workers' performance [Yin and Chen, 2015;Mason and Watts, 2010;Gao et al, 2012;Feng et al, 2014]. Another line of work [Law et al, 2016;Kaufmann et al, 2011] suggests the use of intrinsic behavioral factors to motivate workers. The mechanisms described above were found to be successful in scenarios where the lack of motivation is due to low engagement level of some workers.…”