Microwork platforms as enablers to new ecosystems and business models: the challenge of managing difficult tasks

Abstract

We explore how microwork platforms manage difficult tasks in paid crowdsourcing environments. We argue that as human computation becomes more prevalent, notably in the context of big data ecosystems, microwork platforms might have to evolve and to take a more managerial stance in order to provide the right incentives to online workers to handle difficult tasks. We illustrate this first through a name disambiguation experiment on Amazon Mechanical Turk (AMT), a well-known microwork platform, and second through direct analysis of the dynamics of task execution in a dataset of real microwork projects on AMT. We discuss the emergence of more specialised microwork platforms as an attempt to facilitate a better management of difficult tasks in the context of paid crowdsourcing.

The Life Course: An interdisciplinary framework for broadening the scope of research on crowdwork

“…The third largest group of studies are behavioural experiments (20%). Most use experiments as the only method, but in some cases experiments are combined with (longitudinal) digital behaviour trace data or various activity logs amassed from the platforms (Dalle et al, 2017;Hata et al, 2016;Sodré & Brasileiro, 2017). We term the latter sub-group of experimental studies 'Experiment Plus.'…”

“…30 Williams et al (2019) x x 31 Yin et al (2018) x x 32 Rani & Furrer (2019) x x 33 Sutherland et al (2019) x x 34 Sannon et al (2019) x x 35 d' Eon et al (2019) x x 36 Kinder et al (2019) x x 37 Morris et al (2013) x 38 Schörpf et al (2017) x 39 Dalle et al (2017) x 40 Pongratz (2018) x 41 Ho et al (2015) x 42 Hata et al (2016) x 43 Jain et al (2017) x 44 Naderi (2018) x 45 Graham et al (2017) x 46 Gadiraju et al (2017) x 47 Hirth et al (2011) x 48 Khanna et al (2010) x 49 Jacques et al (2019) x 50 Gerber & Krzywdzinski (2019) x 51 Rani & Furrer (2020) x 52 Newlands & Lutz (2020) x 53 Zyskowski et al (2015) x 54 Gadiraju et al (2015) x 55 Barnes et al (2015) x 56 D' Cruz & Noronha (2016) x 57 Gadiraju et al (2016) x 58 Kost et al (2018) x 59 Posch et al (2018) x 60 Ludec et al (2019) x 61 Margaryan (2019a) x 62 Bucher et al (2019) x 63 Chen et al (2019) x 64 Zhuang & Gadiraju (2019) x 65 Margaryan (2019b) x 66 Gadiraju & Demartini (2019) x 67 Gadiraju et al (2018a) x 68 Ho et al (2018) x 69 Ma et al (2018) x 70 Mubarak et al (2016) x…”

“…• Design of tasks and workflows and their impact on workers' performance (Bucher & Fieseler, 2016;D'Cruz, 2017;Deng & Joshi, 2016;Difallah et al, 2015;Gadiraju, 2015;Gould et al, 2016;Hata et al, 2016;Hirth et al, 2011;Jain et al, 2017;Jiang et al, 2015;Khanna et al, 2010;Kingsley et al, 2015;Law et al, 2016;Naderi, 2018;Retelny et al, 2017;Schörpf et al, 2017;Vashistha et al, 2018) • Nature of tasks and digital traces on platforms (Dalle et al, 2017;Difallah et al, 2015;Hirth et al, 2011;Jain et al, 2017;Lehdonvirta, 2016Lehdonvirta, , 2018Yang et al, 2018) • Overall platform design features, in particular platform policies, incentives and reward systems (Al-Ani & Stumpp, 2016;Dalle et al, 2017;Fieseler et al, 2019;Gerber & Krzywdzinski, 2019;Graham et al, 2017;Ho et al, 2015;Hsieh & Kocielnik, 2016;Jiang et al, 2015;Lehdonvirta, 2018;Pongratz, 2018;Sodré & Brasileiro, 2017;Wang et al, 2017) • Country specifics of platform work, mainly focusing on US and India and comparisons between these (Berg, 2016;D'Cruz, 2017;…”