The future of crowd work

Kittur, Aniket; Nickerson, Jeffrey V.; Bernstein, Michael S.; Gerber, Elizabeth M.; Shaw, Aaron; Zimmerman, John; Lease, Matt; Horton, John J.

doi:10.1145/2441776.2441923

Cited by 890 publications

(799 citation statements)

References 134 publications

Supporting

Mentioning

775

Contrasting

Unclassified

Order By: Relevance

“…Specifically, it is not clear whether their results would transfer to other domains where tasks are not necessarily as short (i.e., under 2 seconds per trial) or atomizable as they were in Komarov et al's work. Repetitive, routine tasks in the crowdsourcing literature are typically split into very short micro-tasks; for more skilled work, multi-hour units of work might be assigned (Kittur et al, 2013). Crowdworkers might typically work for four hours per day on average (Lasecki, Rzeszotarski, Marcus, & Bigham, 2015), individual tasks are usually very much shorter.…”

Section: Online Data Collectionmentioning

confidence: 99%

Home is Where the Lab is: A Comparison of Online and Lab Data From a Time-sensitive Study of Interruption

Gould

Cox

Brumby

et al. 2015

HCj

View full text Add to dashboard Cite

While experiments have been run online for some time with positive results, there are still outstanding questions about the kinds of tasks that can be successfully deployed to remotely situated online participants. Some short tasks, such as menu selection, have produced viable results but these do not represent the gamut of tasks that interest human-computer interaction researchers. In particular, we wondered whether long-lasting, time-sensitive tasks that require continuous concentration could work successfully online, given the confounding effects that might accompany the online deployment of such a task. We ran an archetypal interruption experiment online and in the lab to investigate whether studies that are long-lasting and timesensitive might be more vulnerable to a loss of control than the short, time-insensitive studies that are representative of the majority of previous online studies. Statistical comparisons showed no evidence of performance differences across a number of dimensions. Despite these results, there were issues with data quality that stemmed from participants misunderstanding the task. Our findings suggest that long-lasting experiments using time-sensitive performance measures can be run online but that care must be taken when introducing participants to experimental procedures.

show abstract

Section: Online Data Collectionmentioning

confidence: 99%

Home is Where the Lab is: A Comparison of Online and Lab Data From a Time-sensitive Study of Interruption

Gould

Cox

Brumby

et al. 2015

HCj

View full text Add to dashboard Cite

show abstract

“…The implementation uses the scalsc LSC library, with extensions to model feature trees, labour and team selection, and user populations 3 . Concretely, this comprises:…”

Section: A Prototype Implementationmentioning

confidence: 99%

“…However, there are many cases where where a traditional workflow approach is too rigid to address the dynamic an unpredictable nature of the tasks at hand, and more flexible crowd working systems must be developed [3]. One example of such dynamic systems is the field of social machines-systems where computers carry out the bookkeeping so that humans can concentrate on the creative work [4].…”

Section: Introductionmentioning

confidence: 99%

A Collaboration Model for Community-Based Software Development with Social Machines

Murray-Rust¹,

Šćekić²,

Papapanagiotou³

et al. 2015

EAI Endorsed Transactions on Collaborative Computing

View full text Add to dashboard Cite

Abstract-Today's crowdsourcing systems are predominantly used for processing independent tasks with simplistic coordination. As such, they offer limited support for handling complex, intellectually and organizationally challenging labour types, such as software development. In order to support crowdsourcing of the software development processes, the system needs to enact coordination mechanisms which integrate human creativity with machine support. While workflows can be used to handle highlystructured and predictable labour processes, they are less suitable for software development methodologies where unpredictability is an unavoidable part the process. This is especially true in phases of requirement elicitation and feature development, when both the client and development communities change with time. In this paper we present models and techniques for coordination of human workers in crowdsourced software development environments. The techniques augment the existing Social Compute Unit (SCU) concept-a general framework for management of ad-hoc human worker teams-with versatile coordination protocols expressed in the Lightweight Social Calculus (LSC). This approach allows us to combine coordination and quality constraints with dynamic assessments of software-user's desires, while dynamically choosing appropriate software development coordination models.

show abstract

“…In many crowdsourced tasks, people work as individuals and compete with each other for a prize (Ebner, et al, 2009;Leimeister, et al, 2009). Alternatively, people's work can also be coordinated such that there is collaboration or integration (Kittur, et al, 2013;Michelucci and Dickinson, 2016;Ren, et al, 2014). The most famous crowd collaboration may be Wikipedia, an encyclopedia that anyone with Internet access can edit.…”

Section: Introductionmentioning

confidence: 99%

Combining Ideas in Crowdsourced Idea Generation

Wang

Wang²,

Tao

2017

Foundations of Management

View full text Add to dashboard Cite

Collecting ideas through crowdsourcing has become a common practice for companies to benefit from external ideas and innovate. It is desirable that crowd members build on each other's ideas to achieve synergy. This study proposes and verifies a new method for idea combination which can result in combined ideas that are both novel and useful. The domain-specific knowledge of crowd members does not influence the effectiveness of such idea combination. The new method can be used for collecting highly creative ideas from the crowd. The implications for future research are discussed.

show abstract

The future of crowd work

Cited by 890 publications

References 134 publications

Home is Where the Lab is: A Comparison of Online and Lab Data From a Time-sensitive Study of Interruption

Home is Where the Lab is: A Comparison of Online and Lab Data From a Time-sensitive Study of Interruption

A Collaboration Model for Community-Based Software Development with Social Machines

Combining Ideas in Crowdsourced Idea Generation

Contact Info

Product

Resources

About