Software crowdsourcing reliability: an empirical study on developers behavior

Alelyani, Turki; Yang, Ye

doi:10.1145/2989238.2989245

Cited by 11 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the first interpretation according to Mao et al [21] and Wu et al [38] there are several online platforms available that can currently host software development challenges. Several of these platforms support all types of software development tasks 1 (TopCoder, Bountify) while others support specific tasks 2 such as testing and mobile development (uTest, Test-Birds). According to Mao et al [21] TopCoder is a pioneer for practicing crowdsourced software engineering and the dominant platform when selecting the medium to host contents.…”

Section: Where To Crowdsource?mentioning

confidence: 99%

“…According to Mao et al [21] TopCoder is a pioneer for practicing crowdsourced software engineering and the dominant platform when selecting the medium to host contents. Additionally this platform is used in the majority of research studies performed for determining success factors of crowdsourced software development [1], [2], [19], [36], [38], [39]. However, we were not able to identify any study that compares the efficacy of different platforms.…”

Section: Where To Crowdsource?mentioning

confidence: 99%

“…On the other hand, the number of competitions that run in parallel within a project has a significant negative effect on the crowd's interest in a competition. Alelyani and Yang [1] explored crowd reliability by investigating possible connections between the nature of the crowd and crowdsourced tasks. Data regarding workforce reliability (in terms of complete task submission and thus participation), task registration speed, task completion duration, skills (on programming language knowledge list) and challenge types (on the nature and rewards of each contest) were analyzed and signalized as catalyst factors for success.…”

Section: F How To Crowdsource?mentioning

confidence: 99%

“…According to Dahlander and Piezunka [5] only 10% of crowdsourced contests are able to attract the desired amount of contributions, while in 50% of the contests there is no contribution at all. The success of crowdsourced software development is closely related to the reliability of the participating crowd-workers [1] and their experience [37] and therefore it is important for contest providers to attract the suitable contributors [7]. According to Weidema et al [37] many participants find the crowdsourced tasks to be difficult despite the fact they are of limited scope.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

et al. 2020

View full text Add to dashboard Cite

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

show abstract

Section: Where To Crowdsource?mentioning

confidence: 99%

Section: Where To Crowdsource?mentioning

confidence: 99%

Section: F How To Crowdsource?mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Furthermore, they put forward the trustworthiness of software crowdsourcing to help enterprises get rid of certain risks [26]. Alelyani T and Yang Y [27] discussed the reliability that different workers showed on different tasks, so as to find out certain behavioral characteristics of the crowd developers. Hu Z and Wu W [28,29] studied the offense-defense mechanism of Topcoder.com contests, constructed a game theoretical model, and calculated the Nash equilibrium to deduce these developers' specific behavior decisions.…”

Section: Research On Crowd Workersmentioning

confidence: 99%

A Data-Driven Game Theoretic Strategy for Developers in Software Crowdsourcing: A Case Study

Liao

Zhang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Crowdsourcing has the advantages of being cost-effective and saving time, which is a typical embodiment of collective wisdom and community workers’ collaborative development. However, this development paradigm of software crowdsourcing has not been used widely. A very important reason is that requesters have limited knowledge about crowd workers’ professional skills and qualities. Another reason is that the crowd workers in the competition cannot get the appropriate reward, which affects their motivation. To solve this problem, this paper proposes a method of maximizing reward based on the crowdsourcing ability of workers, they can choose tasks according to their own abilities to obtain appropriate bonuses. Our method includes two steps: Firstly, it puts forward a method to evaluate the crowd workers’ ability, then it analyzes the intensity of competition for tasks at Topcoder.com—an open community crowdsourcing platform—on the basis of the workers’ crowdsourcing ability; secondly, it follows dynamic programming ideas and builds game models under complete information in different cases, offering a strategy of reward maximization for workers by solving a mixed-strategy Nash equilibrium. This paper employs crowdsourcing data from Topcoder.com to carry out experiments. The experimental results show that the distribution of workers’ crowdsourcing ability is uneven, and to some extent it can show the activity degree of crowdsourcing tasks. Meanwhile, according to the strategy of reward maximization, a crowd worker can get the theoretically maximum reward.

show abstract

Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature review

ZhenYing

KhanAbdullah

Nazir

et al. 2021

J Software Evolu Process

View full text Add to dashboard Cite

Crowdsourcing is simply the outsourcing of different tasks or work to a diverse group of individuals in an open call for the purpose of utilizing human intelligence.Crowdsourcing nowadays used to support and enhance software engineering in different aspects. In this proposed study, a systematic literature review was conducted for the last 10 years from 2010 to 2019. During the filtering process, a total of 120 relevant studies have been identified, and then the most relevant 70 studies were selected to include as part of the current study. The proposed study shows the effect of task assignment in crowdsourcing, such as if a task is assigned to an appropriate worker or an inappropriate worker, what will be the consequences.The study also highlights crowdsourcing usage in the field of software engineering.All the existing task assignment methods used for assigning the task to make crowdsourcing activity more effective have been analyzed. The study also highlights all the available crowdsourcing platforms used for a variety of task to be performed.The study concludes by identifying the issues regarding the task assignment and to specify the methods for enhancement in the assignment of tasks for future research in crowdsourcing.

show abstract

Software crowdsourcing reliability: an empirical study on developers behavior

Cited by 11 publications

References 13 publications

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

A Data-Driven Game Theoretic Strategy for Developers in Software Crowdsourcing: A Case Study

Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature review

Contact Info

Product

Resources

About