A systematic literature review on crowdsourcing in software engineering

Sarı, Aslı; Tosun, Ayşe; Alptekin, Gülfem Işıklar

doi:10.1016/j.jss.2019.04.027

Cited by 54 publications

(52 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Trust is the key element in choosing employees for a task [19,[27][28][29]. In different practices such as in the discovery of truth and in selection of workers, trust plays a significant role [30].…”

Section: Crowd Selection Based On Trustworthinessmentioning

confidence: 99%

“…Expertise [11,[26][27][28][32][33][34][35][36][37] level of the crowd is estimated by expertise filtering. According to the expertise level, a right crowd for a task is selected [33,34].…”

Section: Crowd Selection On the Basis Of Expertise Filteringmentioning

confidence: 99%

“…Crowd selection is a complex problem in which the skills and knowledge of huge crowd workers are matched with the requirements of a job [26]. Crowd selection or formation is an optimization problem that involves all types of approaches to build a crowd group to whom various tasks will be offered [27]. Various features of crowd workers were identified from existing studies with the aim to use it for crowd selection in crowdsourcing.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multicriteria‐Based Crowd Selection Using Ant Colony Optimization

et al. 2021

View full text Add to dashboard Cite

Internet-enabled technologies have provided a way for people to communicate and collaborate with each other. The collaboration and communication made crowdsourcing an efficient and effective activity. Crowdsourcing is a modern paradigm that employs cheap labors (crowd) for accomplishing different types of tasks. The task is usually posted online as an open call, and members of the crowd self-select a task to be carried out. Crowdsourcing involves initiators or crowdsourcers (an entity usually a person or an organization who initiate the crowdsourcing process and seek out the ability of crowd for a task), the crowd (online participant who is a having a particular background, qualification, and experience for accomplishing task in crowdsourcing activity), crowdsourcing task (the activity in which the crowd contribute), the process (how the activity is carried out), and the crowdsourcing platform (software or market place) where requesters offer various tasks and crowd workers complete these tasks. As the crowdsourcing is carried out in the online environment, it gives rise to certain challenges. The major problem is the selection of crowd that is becoming a challenging issue with the growth in crowdsourcing popularity. Crowd selection has been significantly investigated in crowdsourcing processes. Nonetheless, it has observed that the selection is based only on a single feature of the crowd worker which was not sufficient for appropriate crowd selection. For addressing the problem of crowd selection, a novel “ant colony optimization-based crowd selection method” (ACO-CS) is presented in this paper that selects a crowd worker based on multicriteria features. By utilizing the proposed model, the efficiency and effectiveness of crowdsourcing activity will be increased.

show abstract

Section: Crowd Selection Based On Trustworthinessmentioning

confidence: 99%

“…Expertise [11,[26][27][28][32][33][34][35][36][37] level of the crowd is estimated by expertise filtering. According to the expertise level, a right crowd for a task is selected [33,34].…”

Section: Crowd Selection On the Basis Of Expertise Filteringmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multicriteria‐Based Crowd Selection Using Ant Colony Optimization

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Sarı et al [26] mention that crowdsourcing has been applied to various process areas in software engineering with coding tasks being the most popular one. Task selection of the software development activities crowdsourced is a great challenge for stakeholders as micro-tasks are more easy to complete according to Latoza and Van Der Hoek [17], but bigger tasks may under the right environment leverage the most the power of the crowd.…”

Section: B What To Crowdsource?mentioning

confidence: 99%

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

et al. 2020

View full text Add to dashboard Cite

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

show abstract

“…Continuously mounting volume of data across social media and community websites has become a rich and highly potential knowledge source for a large group of users. Now a days, if people need to acquire knowledge about new subject or want to solve some particular problem, they look towards fastest, reliable and to the point information that address their needs [1][2] [3]. People very often tend to utilize pedagogical values of social media like web communities, online platforms, Community Questioning Answering (CQA), generally known as questioning answering websites, crawling large amount of data from a large array of geographically distributed users with variety of expertise.…”

Section: Introductionmentioning

confidence: 99%

Educational Data Classification Framework for Community Pedagogical Content Management using Data Mining

Mushtaq¹,

Siddique²,

Malik³

et al. 2019

ijacsa

View full text Add to dashboard Cite

Recent years witness the significant surge in awareness and exploitation of social media especially community Question and Answer (Q&A) websites by academicians and professionals. These sites are, large repositories of vast data, pawing ways to new avenues for research through applications of data mining and data analysis by investigation of trending topics and the topics of most attention of users. Educational Data Mining (EDM) techniques can be used to unveil potential of Community Q&A websites. Conventional Educational Data Mining approaches are concerned with generation of data through systematic ways and mined it for knowledge discovery to improve educational processes. This paper gives a novel idea to explore already generated data through millions of users having variety of expertise in their particular domains across a common platform like StackOverFlow (SO), a community Q&A website where users post questions and receive answers about particular problems. This study presents an EDM framework to classify community data into Software Engineering subjects. The framework classifies the SO posts according to the academic courses along with their best solutions to accommodate learners. Moreover, it gives teachers, instructors, educators and other EDM stakeholders an insight to pay more attention and focus on commonly occurring subject related problems and to design and manage of their courses delivery and teaching accordingly. The data mining framework performs preprocessing of data using NLP techniques and apply machine learning algorithms to classify data. Amongst all, SVM gives better performs with 72.06% accuracy. Evaluation measures like precision, recall and F-1 score also used to evaluate the best performing classifier.

show abstract

A systematic literature review on crowdsourcing in software engineering

Cited by 54 publications

References 8 publications

Multicriteria‐Based Crowd Selection Using Ant Colony Optimization

Multicriteria‐Based Crowd Selection Using Ant Colony Optimization

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Educational Data Classification Framework for Community Pedagogical Content Management using Data Mining

Contact Info

Product

Resources

About