Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing

Qiu, Xiaoyu; Liu, Luobin; Chen, Wuhui; Hong, Zicong; Zheng, Zibin

doi:10.1109/tvt.2019.2924015

Cited by 238 publications

(91 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among all these AI technologies, Q-learning and its derivate, DQN, are in the spotlight. For example, [42] designs a Q-learning-based algorithm for computation offloading. Concretely, it formulates the computation offloading problem as a non-cooperative game in multi-user multi-server edge computing systems and proves that Nash Equilibrium exists.…”

Section: Problem Definition Model Construction Algorithm Designmentioning

confidence: 99%

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Deng

Zhao

Fang

et al. 2020

IEEE Internet Things J.

588

209

View full text Add to dashboard Cite

Along with the deepening development in communication technologies and the surge of mobile devices, a brandnew computation paradigm, Edge Computing, is surging in popularity. Meanwhile, Artificial Intelligence (AI) applications are thriving with the breakthroughs in deep learning and the upgrade of hardware architectures. Billions of bytes of data, generated at the network edge, put great demands on data processing and structural optimization. Therefore, there exists a strong demand to integrate Edge Computing and AI, which gives birth to Edge Intelligence. In this article, we divide Edge Intelligence into AI for edge (Intelligence-enabled Edge Computing) and AI on edge (Artificial Intelligence on Edge). The former focuses on providing a more optimal solution to the key concerns in Edge Computing with the help of popular and effective AI technologies while the latter studies how to carry out the entire process of building AI models, i.e., model training and inference, on edge. This article focuses on giving insights into this new inter-disciplinary field from a broader vision and perspective. It discusses the core concepts and the research road-map, which should provide the necessary background for potential future research programs in Edge Intelligence.

show abstract

Section: Problem Definition Model Construction Algorithm Designmentioning

confidence: 99%

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Deng

Zhao

Fang

et al. 2020

IEEE Internet Things J.

588

209

View full text Add to dashboard Cite

show abstract

“…P π(s) s,st+1 denotes the transition probability unknown in reality, and π(s) is the action generated under a specific policy. Based on (17) and (18), the cost functions (11), (12) can be rewritten as:…”

Section: A Bellman Equationmentioning

confidence: 99%

“…In order to achieve the ability to learn automatically, we design the updating steps as following [35]: 1) evaluating result: obtain V i,π (s) and V total,π (q) according to (17) and (18) based on the policy π for all status.…”

Section: A Bellman Equationmentioning

confidence: 99%

“…Therefore, more and more communication scientists focus on developing the potential of ML for some tricky communication problems. As an important branch, deep reinforcement learning (RL) has attracted great attention recently [17]- [22]. The authors in [17] investigate the computation offloading problem in blockchain empowered mobile edge computing system, where the deep RL algorithm is applied to the computing offloading decision-making process.…”

Section: Introductionmentioning

confidence: 99%

“…As an important branch, deep reinforcement learning (RL) has attracted great attention recently [17]- [22]. The authors in [17] investigate the computation offloading problem in blockchain empowered mobile edge computing system, where the deep RL algorithm is applied to the computing offloading decision-making process. Moreover, deep RL is also applied in unmanned aerial vehicle autonomous target searching in a complex disaster scene [18], where the superior ability on dynamic programming of deep RL can be observed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Energy Minimization in D2D-Assisted Cache-Enabled Internet of Things: A Deep Reinforcement Learning Approach

Tang

Zhang

et al. 2020

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Mobile edge caching (MEC) and device to device (D2D) communications are two potential technologies to resolve traffic overload problems in internet of things (IoT). Previous works usually investigate them separately with MEC for traffic offloading and D2D for information transmission. In this paper, a joint framework consisting of MEC and cache-enabled D2D communications is proposed to minimize the energy cost of systematic traffic transmission, where file popularity and user preference are the critical criteria for small base stations (SBSs) and user devices, respectively. Under this framework, we propose a novel caching strategy where Markov decision process (MDP) is applied to model the requesting behaviours. A novel scheme based on reinforcement learning (RL) is proposed to reveal the popularity of files as well as users' preference. In particular, Q-learning (QL) algorithm and deep Q-network (DQN) algorithm are respectively applied to user devices and SBS due to different complexities of status. To save the energy cost of systematic traffic transmission, users acquire partial traffic through D2D communications based on the cached contents and user distribution. Taking the memory limits, D2D available files and status changing into consideration, the proposed RL algorithm enables user devices and SBS to prefetch the optimal files while learning, which can reduce the energy cost significantly. Simulation results demonstrate the superior energy saving performance of the proposed RL-based algorithm over other existing methods under various conditions.

show abstract

A comprehensive review of blockchain applications in industrial Internet of Things and supply chain systems

Malik

Alkhatib

Sun

et al. 2021

Appl Stoch Models Bus & Ind

View full text Add to dashboard Cite

The industrial Internet of Things (IIoT) and supply chain systems have become one of the most important technologies to positively impact our daily life activities. Blockchain technology has recently gained popularity and it is increasingly being employed in a large number of applications and industries. The use cases for blockchain are growing rapidly in different fields and domains. Combining blockchain into IIoT and in our supply chain logistics could provide many solutions to the critical issues that current systems face today. Blockchain features such as immutability, decentralization, and data encryption are useful in solving the shortcomings of IIoT and the current supply chain systems security and integrity. This article presents a comprehensive review of blockchain applications in IIoT and supply chain systems. This article also focuses on the security aspects of blockchain integration within industry applications with the support of the emerging edge computing technology.

show abstract

Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing

Cited by 238 publications

References 39 publications

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Energy Minimization in D2D-Assisted Cache-Enabled Internet of Things: A Deep Reinforcement Learning Approach

A comprehensive review of blockchain applications in industrial Internet of Things and supply chain systems

Contact Info

Product

Resources

About