A trust-aware task allocation method using deep q-learning for uncertain mobile crowdsourcing

Abstract

IntroductionWith the advancing technology of mobile devices with numerous built-in sensors, mobile crowdsourcing has recently emerged as a new collaboration paradigm in numerous intelligent mobile information systems [1]. The existing mobile crowdsourcing has applications in numerous domains including urban planning, traffic monitoring, ride sharing, environmental monitoring and intelligent disaster response [2]. Mobile crowdsourcing is a combination of spatial crowdsourcing and smart phone technology that employs mobile workers to perform certain tasks in a specific AbstractMobile crowdsourcing has emerged as a promising collaboration paradigm in which each spatial task requires a set of mobile workers in near vicinity to the target location. Considering the desired privacy of the participating mobile devices, trust is considered to be an important factor to enable effective collaboration in mobile crowdsourcing. The main impediment to the success of mobile crowdsourcing is the allocation of trustworthy mobile workers to nearby spatial tasks for collaboration. This process becomes substantially more challenging for large-scale online spatial task allocations in uncertain mobile crowdsourcing systems. The uncertainty can mislead the task allocation, resulting in performance degradation. Moreover, the large-scale nature of real-world crowdsourcing poses a considerable challenge to spatial task allocation in uncertain environments. To address the aforementioned challenges, first, an optimization problem of mobile crowdsourcing task allocation is formulated to maximize the trustworthiness of workers and minimize movement distance costs. Second, for the uncertain crowdsourcing scenario, a Markov decision process-based mobile crowdsourcing model (MCMDP) is formulated to illustrate the dynamic trust-aware task allocation problem. Third, to solve large-scale MCMDP problems in a stable manner, this study proposes an improved deep Q-learning-based trust-aware task allocation (ImprovedDQL-TTA) algorithm that combines trust-aware task allocation and deep Q-learning as an improvement over the uncertain mobile crowdsourcing systems. Finally, experimental results illustrate that the ImprovedDQL-TTA algorithm can stably converge in a number of training iterations. Compared with the reference algorithm, our proposed algorithm achieves effective solutions on the experimental data sets. which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Exploration with Multiple Random ε-Buffers in Off-Policy Deep Reinforcement Learning

“…Therefore, our proposed method can advance the exploration performance of εgreedy, because multiple mini-batches do not follow a local minimum [9,10]. deep RL, as in a recent study [3][4][5][6][7], we acquire better results. Moreover, we can attempt a different deep neural network and hyper-parameter settings to demonstrate that our proposed model can enhance the exploration without specific domain information.…”

“…A combination of RL and nonlinear function approximates, such as deep neural networks, helps in automating decision making and control issues. However, it can also be challenging in terms of stability and convergence [2] in real-time online learning, such as in the detection of network security through the verification of whether the network is "normal or anomalous" [3][4][5][6][7]. Moreover, its widespread adaptation in the real world, such as with robotic arms, is difficult because of sample complexity and heavy dependence on certain hyper-parameters, such as exploration constants [3][4][5][6][7].…”

“…The proposed deep RL model is compatible with the discrete action as well as continuous control, symmetrically. Therefore, we can expect better prediction accuracy in real-time online learning, such as detecting network intrusions by verifying whether the network is "normal or anomalous" [3][4][5][6][7]. The concept of our proposed multiple ε-buffers is extremely simple.…”

Reinforcement Learning in Dynamic Task Scheduling: A Review

“…Value-iteration methods are often carried out off-policy, meaning that the policy used to generate behavior for training data can be unrelated to the policy being evaluated and improved, called the estimation policy [11,12]. Popular value-iteration methods used in dynamic task scheduling are Q-Learning [7,9,10,[15][16][17] and Deep Q-Network (DQN) [3,8,[18][19][20]. Apart from these two, Greedy methods [19], Monte Carlo Methods [21] and Temporal Difference (TD) Learning [22,23] also have been used.…”

“…The research done by Sun and Tan [18] focuses on the trust-aware task allocation (TTA) optimization problem / dynamic trust-aware task allocation problem of mobile crowdsourcing systems. They have done the experiment in an uncertain crowdsourcing environment and have researched on how to maximize the trust score and minimize the travel distance cost [18]. Here, the TTA optimization problem aims at maximizing trust score and reducing the travel distance cost.…”