The Effects of Pay-to-Quit Incentives on Crowdworker Task Quality

Harris, Constance M.

doi:10.1145/2675133.2675185

Cited by 8 publications

(10 citation statements)

References 28 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Brabham, 2010Brabham, , 2008bKaufmann et al, 2011;Zhao and Zhu, 2014b;Zheng et al, 2011) and incentive design (e.g. Harris et al, 2015;Leimeister et al, 2009;Straub et al, 2015). Studies have shown that a wide variety of reasons and motivations, ranging from intrinsic to extrinsic, lead people to participate in crowdsourcing and related online work and economic coordination Kaufmann et al, 2011;Straub et al, 2015;Zhao and Zhu, 2014b;Zheng et al, 2011).…”

Section: Gamificationmentioning

confidence: 99%

“…To date, it is unclear for which crowdsourcing system type, crowdsourcee type, and task type the use of gamification is more beneficial compared to financial incentives, or when the combination of the two is the best approach. Future research should compare different incentive mechanisms (see Straub et al, 2015;Harris et al, 2015) and should consider contextual factors and user characteristics. The findings summarized in Table 6 Shen et al, 2009;X.-L. Shen et al, 2014).…”

Section: Thematic Agendamentioning

confidence: 99%

See 1 more Smart Citation

Gamified crowdsourcing: Conceptualization, literature review, and future agenda

Morschheuser

Hamari

Koivisto

et al. 2017

International Journal of Human-Computer Studies

242

180

View full text Add to dashboard Cite

Two parallel phenomena are gaining attention in human-computer interaction research: gamification and crowdsourcing. Because crowdsourcing's success depends on a mass of motivated crowdsourcees, crowdsourcing platforms have increasingly been imbued with motivational design features borrowed from games; a practice often called gamification. While the body of literature and knowledge of the phenomenon have begun to accumulate, we still lack a comprehensive and systematic understanding of conceptual foundations, knowledge of how gamification is used in crowdsourcing, and whether it is effective. We first provide a conceptual framework for gamified crowdsourcing systems in order to understand and conceptualize the key aspects of the phenomenon. The paper's main contributions are derived through a systematic literature review that investigates how gamification has been examined in different types of crowdsourcing in a variety of domains. This meticulous mapping, which focuses on all aspects in our framework, enables us to infer what kinds of gamification efforts are effective in different crowdsourcing approaches as well as to point to a number of research gaps and lay out future research directions for gamified crowdsourcing systems. Overall, the results indicate that gamification has been an effective approach for increasing crowdsourcing participation and the quality of the crowdsourced work; however, differences exist between different types of crowdsourcing: the research conducted in the context of crowdsourcing of homogenous tasks has most commonly used simple gamification implementations, such as points and leaderboards, whereas crowdsourcing implementations that seek diverse and creative contributions employ gamification with a richer set of mechanics.

show abstract

Section: Gamificationmentioning

confidence: 99%

Section: Thematic Agendamentioning

confidence: 99%

Gamified crowdsourcing: Conceptualization, literature review, and future agenda

Morschheuser

Hamari

Koivisto

et al. 2017

International Journal of Human-Computer Studies

242

180

View full text Add to dashboard Cite

show abstract

“…They highlighted the consequent difference in the crowdsourced task results [31]. Along the same line, in [28] authors showed that rewarding workers when they quit their participation in a batch of HITs allows to filter out low-quality workers early, thus retaining only highly accurate workers. Recently, Findlater et al showed that results of online HCI experiments are similar to those achieved in the lab for desktop interactions, but this was less so in the case of mobile devices [20].…”

Section: Worker Differences and Participation Biasmentioning

confidence: 91%

Modus Operandi of Crowd Workers

Gadiraju

Checco

Gupta

et al. 2017

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that of work environments; defined as the hardware and software affordances at the disposal of crowd workers which are used to complete microtasks on crowdsourcing platforms. In this paper, we reveal the significant role of work environments in the shaping of crowd work. First, through a pilot study surveying the good and bad experiences workers had with UI elements in crowd work, we revealed the typical issues workers face. Based on these findings, we then deployed over 100 distinct microtasks on CrowdFlower, addressing workers in India and USA in two identical batches. These tasks emulate the good and bad UI element designs that characterize crowdsourcing microtasks. We recorded hardware specifics such as CPU speed and device type, apart from software specifics including the browsers used to complete tasks, operating systems on the device, and other properties that define the work environments of crowd workers. Our findings indicate that crowd workers are embedded in a variety of work environments which influence the quality of work produced. To confirm and validate our data-driven findings we then carried out semi-structured interviews with a sample of Indian and American crowd workers from this platform. Depending on the design of UI elements in microtasks, we found that some work environments support crowd workers more than others. Based on our overall findings resulting from all the three studies, we introduce ModOp, a tool that helps to design crowdsourcing microtasks that are suitable for diverse crowd work environments. We empirically show that the use of ModOp results in reducing the cognitive load of workers, thereby improving their user experience without effecting the accuracy or task completion time.

show abstract

“…16 We also note an increased academic interest in pay-to-quit schemes, see e.g. the experimental study by Harris (2015). 15 Note that Switzerland and other countries offer several forms of return assistance to refugees and other migrants (see e.g.…”

Section: Appendixmentioning

confidence: 97%

Paid to Quit

Dur

Schmittdiel

2019

De Economist

View full text Add to dashboard Cite

Inspired by a recent observation about an online retail company, this paper explains why a firm may find it optimal to offer an exit bonus to recent hires so as to induce self-selection. We study a double adverse selection problem, in which the principal can neither observe agents' commitment to the job nor their intrinsic motivation. A steep wage-tenure profile deters uncommitted agents from applying. An exit bonus can stimulate that-among the committed agents-those who discovered that they are not intrinsically motivated for the job discontinue employment with the principal. Our key findings are that offering an exit bonus increases profits when the first adverse selection problem is sufficiently severe compared to the second and that the exit bonus needs to come as a surprise for the agents in order to function well.

show abstract

The Effects of Pay-to-Quit Incentives on Crowdworker Task Quality

Cited by 8 publications

References 28 publications

Gamified crowdsourcing: Conceptualization, literature review, and future agenda

Gamified crowdsourcing: Conceptualization, literature review, and future agenda

Modus Operandi of Crowd Workers

Paid to Quit

Contact Info

Product

Resources

About