Utilization of synergetic human-machine clouds: a big data cleaning case

Abstract

Cloud computing and crowdsourcing are growing trends in IT. Combining the strengths of both machine and human clouds within a hybrid design enables us to overcome certain problems and achieve efficiencies. In this paper we present a case in which we developed a hybrid, throw-away prototype software system to solve a big data cleaning problem in which we corrected and normalized a data set of 53,822 academic publication records. The first step in our solution consists of utilization of external DOI query web services to label the records with matching DOIs. Then we used customized string similarity calculation algorithms based on Levensthein Distance and Jaccard Index to grade the similarity between records. Finally we used crowdsourcing to identify duplicates among the residual record set consisting of similar yet not identical records. We consider this proof of concept to be successful and report that we achieved certain results that we could not have achieved by using either human or machine clouds alone.

Crowd-Enhanced Cloud Services: Issues and Directions

“…The same trend is emerging in the cloud services as well [11,12]. Therefore, in this section it is tried to present justifications on cloud-crowd computing by answering the following questions:…”

“…The goal of this research as a proof of concept was to show that crowdsourcing can be used effectively and efficiently as part of software engineering practices [11].…”

“…Nowadays, crowds are increasingly employed in software and used alongside with the computers to solve problems. The same trend is emerging in the cloud services as well [11,12]. Crowd-enhanced cloud which is the goal of this paper is a type of distributed system created as a result of injecting crowdsourcing into cloud computing at the lowest level of cloud hierarchy a.k.a resource pool.…”

A multi-layered desires based framework to detect evolving non-functional requirements of users

“…Since the users' self-reported desires served as the ground truth for validating desires inference accuracy, it was critical to filter noisy records when participants did not perform conscientiously. Afshan et al used Gold standard questions to assure participants payed continuous attentions during a survey [36,31]. We follow similar rules to remove data noise, as described as follows:…”