CrowdWT

Abstract

Crowdsourcing is a relatively inexpensive and efficient mechanism to collect annotations of data from the open Internet. Crowdsourcing workers are paid for the provided annotations, but the task requester usually has a limited budget. It is desirable to wisely assign the appropriate task to the right workers, so the overall annotation quality is maximized whilst the cost is reduced. In this paper, we propose a novel task assignment strategy (CrowdWT) to capture the complex interactions between tasks and workers, and properly assign tasks to workers. CrowdWT first develops a Worker Bias Model (WBM) to jointly model the worker's bias, the ground truths of tasks, and the task features. WBM constructs a mapping between task features and worker annotations to dynamically assign the task to a group of workers, who are more likely to give correct annotations for the task. CrowdWT further introduces a Task Difficulty Model (TDM), which builds a Kernel ridge regressor based on task features to quantify the intrinsic difficulty of tasks and thus to assign the difficult tasks to more reliable workers. Finally, CrowdWT combines WBM and TDM into a unified model to dynamically assign tasks to a group of workers, recall more reliable even expert workers to annotate the difficult tasks.Our experimental results on two real-world datasets and two semi-synthetic datasets show that CrowdWT achieves high-quality answers within a limited budget, and has the best performance against competitive methods.

Quantity sensitive task allocation based on improved whale optimization algorithm in crowdsensing system

“…Choose the air curtain attack method, and update the individual position according to formula ( 16 According to formula (13) and formula (14), perform local dithering on the optimal individual…”

“…In addition to considering the position match between the user and the task, Zhao et al 13 started from the homogeneity of humans and applied the relationship between friends in social networks to task allocation scenarios, and proposed a task allocation algorithm based on friend relationships. By comparing this algorithm with other algorithms on the synthetic dataset and the real dataset named Yelp, the effectiveness and applicability of the allocation mechanism were verified; Tu et al 14 hoped to assign it to appropriate user according to the characteristics of the task, so as to reduce the cost while increasing the quality of the perceived data. Based on this, a novel task allocation strategy is proposed, which can dynamically allocate tasks to a group of users by capturing the complex interaction between tasks and users; Zhu et al 15 found that when faced with some complex perception tasks, users' completion is generally low.…”

Incentive-Boosted Federated Crowdsourcing

“…FL enables distributed computing nodes to collaboratively train models without exposing their sensitive data, thus realizing privacypreserving model training with little loss (or even no loss) of model performance (Wang, Yu, and Han 2020;Yang et al 2019). The crowdsourcing system typically outsources data collection tasks to Internet workers and then aggregates and analyzes the sensing data (Capponi et al 2019;Gummidi, Xie, and Pedersen 2019;Tu et al 2020;Yu et al 2020a). Nevertheless, the centralized platform is generally untrusted and may leak workers' private information.…”