Pattern-Based Specification of Crowdsourcing Applications

Abstract

In many crowd-based applications, the interaction with performers is decomposed in several tasks that, collectively, produce the desired results. Tasks interactions give rise to arbitrarily complex workflows. In this paper we propose methods and tools for designing crowd-based workflows as interacting tasks. We describe the modelling concepts that are useful in such framework, including typical workflow patterns, whose function is to decompose a cognitively complex task into simple interacting tasks so that the complex task is co-operatively solved. We then discuss how workflows and patterns are managed by CrowdSearcher, a system for designing, deploying and monitoring applications on top of crowd-based systems, including social networks and crowdsourcing platforms. Tasks performed by humans consist of simple operations which apply to homogeneous objects; the complexity of aggregating and interpreting task results is embodied within the framework. We show our approach at work on a validation scenario and we report quantitative findings, which highlight the effect of workflow design on the final result

BPMN Task Instance Streaming for Efficient Micro-task Crowdsourcing Processes

“…Each step in the resulting process is a crowd task performed on Mechanical Turk. CrowdSearcher [4] is a crowdsourcing system that leverages on reusable design patterns and on tasks performed by machines or people on crowdsourcing platforms or on Facebook.…”

Programming Model Elements for Hybrid Collaborative Adaptive Systems

“…Composition state: The composition process calculates feasible task execution plans, consisting of ordered activities (steps) required to process the given task and associated performer peers. Generation of execution plans is usually a task-specific, non-trivial problem involving advanced planning and constraint satisfaction algorithms, going well beyond the scope of this paper; the description of the currently offered composition algorithms can be found in [9] and here 4 . From the programming model's perspective, however, it suffices to know the required inputs and outputs of this state: the input is the 'provisioned' collective from the previous state, while the output is a list of collectives 'negotiables', associated with composed execution plans, which get passed on to the following state.…”

“…We have recently witnessed the evolution of conventional social computing and appearance of novel types of sociotechnical systems, attempting to leverage human experts for more intellectually challenging tasks [1,2,3,4,5]. These types of systems are opening up the possibilities for novel forms of interaction, collaboration and organization of labor where humans and computers complement each other.…”

Transforming Collaboration Structures into Deployable Informal Processes

“…These approaches focus primarily on achieving sufficient quality at low costs when distributing, collecting, and filtering massive amounts of tasks. Interaction among task workers is not foreseen; all dependencies are modeled as task-centric workflow patterns [9]. Brambilla and Mauri integrate social network-centric actions into web applications via social primitives [11].…”