Handling query answering in crowdsourcing systems: A belief function-based approach

Koulougli, Dalila; Hadjali, Allel; Rassoul, Idir

doi:10.1109/nafips.2016.7851590

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The CASCAD method for modeling and aggregating contributions from crowdsourcing platforms has been defined by Koulougli et al (2016). In order to apply CASCAD, the contributor must perform a qualification test before the campaign to determine his expertise according to the following scale.…”

Section: Answer Aggregationmentioning

confidence: 99%

Estimation of the qualification and behavior of a contributor and aggregation of his answers in a crowdsourcing context

Thierry¹,

Martin²,

Dubois³

et al. 2023

Preprint

View full text Add to dashboard Cite

Crowdsourcing is the outsourcing of tasks to a crowd of contributors on a dedicated platform. The crowd on these platforms is very diversified and includes various profiles of contributors which generates data of uneven quality. However, majority voting, which is the aggregating method commonly used in platforms, gives equal weight to each contribution. To overcome this problem, we propose a method, MONITOR, which estimates the contributor's profile and aggregates the collected data by taking into account their possible imperfections thanks to the theory of belief functions. To do so, MONITOR starts by estimating the profile of the contributor through his qualification for the task and his behavior.Crowdsourcing campaigns have been carried out to collect the necessary data to test MONITOR on real data in order to compare it to existing approaches.The results of the experiments show that thanks to the use of the MONITOR method, we obtain a better rate of correct answer after aggregation of the contributions compared to the majority voting. Our contributions in this article are for the first time the proposal of a model that takes into account both the qualification of the contributor and his behavior in the estimation of his profile.For the second one, the weakening and the aggregation of the answers according to the estimated profiles.

show abstract

Section: Answer Aggregationmentioning

confidence: 99%

Estimation of the qualification and behavior of a contributor and aggregation of his answers in a crowdsourcing context

Thierry¹,

Martin²,

Dubois³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…We have proposed a modeling of some data by belief functions but it is possible to go further by using them for example to estimate the expertise of the contributor as do [7]. The data can also be used to compare a probabilistic approach to belief functions [6].…”

Section: Belief Functions From the Datamentioning

confidence: 99%

Real Bird Dataset with Imprecise and Uncertain Values

Thierry

Hoarau

Martin

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The theory of belief functions allows the fusion of imperfect data from different sources. Unfortunately, few real, imprecise and uncertain datasets exist to test approaches using belief functions. We have built real birds datasets thanks to the collection of numerous human contributions that we make available to the scientific community. The interest of our datasets is that they are made of human contributions, thus the information is therefore naturally uncertain and imprecise. These imperfections are given directly by the persons. This article presents the data and their collection through crowdsourcing and how to obtain belief functions from the data.

show abstract

“…Pour la méthode cascade de Koulougli et al (2016), un test de qualification est réalisé par le contributeur avant l'exécution de la tâche afin de déterminer son profil : "ignorant, peu compétent, moyennement compétent, compétent, expert". À ce profil est associé un coefficient d'affaiblissement allant de 0 (ignorant) à 1 (expert).…”

Section: éTat De L'artunclassified

Modelisation de l'incertitude et de l'imprecision de donnees de crowdsourcing : MONITOR

Thierry,

Dubois,

Gall

et al. 2020

Preprint

View full text Add to dashboard Cite

http://www-druid.irisa.fr Résumé. Le crowdsourcing se définit par l'externalisation de tâches à une foule de contributeurs. La foule est très diversifiée sur ces plateformes et inclut des contributeurs malveillants attirés par la rémunération des tâches et ne les réalisant pas consciencieusement. Il est essentiel d'identifier ces contributeurs afin de ne pas considérer leurs réponses. Tous les contributeurs n'ayant pas les mêmes aptitudes pour une tâche, il semble pertinent d'accorder un poids à leurs réponses en fonction de leurs qualifications. Ce papier, publié lors de la conférence ICTAI 2019, propose une méthode, MONITOR, pour l'estimation du profil du contributeur et l'agrégation des réponses par la théorie des fonctions de croyance.

show abstract

Handling query answering in crowdsourcing systems: A belief function-based approach

Cited by 7 publications

References 11 publications

Estimation of the qualification and behavior of a contributor and aggregation of his answers in a crowdsourcing context

Estimation of the qualification and behavior of a contributor and aggregation of his answers in a crowdsourcing context

Real Bird Dataset with Imprecise and Uncertain Values

Modelisation de l'incertitude et de l'imprecision de donnees de crowdsourcing : MONITOR

Contact Info

Product

Resources

About