An Optimized Resource Scheduling Strategy for Hadoop Speculative Execution Based on Non-cooperative Game Schemes

Jiang, Yinghang; Liu, Qi; Dannah, Williams; Jin, Dandan; Liu, Xiaodong; Sun, Mingxun

doi:10.32604/cmc.2020.04604

Cited by 5 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enhance coverage in WSNs, local information‐based topology control (LITC) algorithm and mobile sensor non‐cooperative game model is proposed using non‐cooperative game theory 34 . Optimized resource scheduling strategy for speculative execution (ORSE) algorithm is proposed using the non‐cooperative game strategy for scheduling of resources of straggling tasks 35 …”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A game theory‐based clustering and multi‐hop routing scheme in wireless sensor networks for energy minimization

Gupta

Aulakh

2022

Int J Communication

View full text Add to dashboard Cite

Summary Energy consumption in wireless sensor network (WSN) is one of the important issues as tiny sensor nodes are equipped with a non‐replaceable batteries. Clustering is one of the effective means to prolong the network lifetime in WSN by reducing the number of packets to be sent from cluster heads (CHs) to base station (BS). Recently, game theory techniques have been applied in the clustering process for optimal CH election at each round. However, none of them have considered the energy parameter in their game strategy and multihop routing for data transmission. In this manuscript, an energy‐efficient game‐theoretic clustering and multi‐hop routing scheme (EGCR) has been proposed where the non‐cooperative approach of game theory is used to elect optimal CHs. In the proposed game theoretic framework, each node behaves in a selfish manner while considering the residual energy of nodes and focuses on increasing the overall lifetime of the network. Nash equilibrium is determined among a set of pure and mixed strategies, and then the expected payoff of a node is determined for Nash equilibria. Apart from the energy‐efficient clustering, the EGCR also proposes a multi‐hop routing solution for data transmission that further enhances the lifetime of whole network. In the multi‐hop routing approach, inter‐cluster and intra‐cluster routing techniques are proposed to identify optimal routes from nodes to CHs and from CHs to BS by using various factors such as node centrality, distance from the node to BS, and residual energy. The proposed algorithm is compared with traditional clustering schemes like low‐energy adaptive clustering hierarchy (LEACH) and efficient energy‐aware game theory‐based clustering (EEGC) and also with existing game theory‐based algorithms such as clustered routing for selfish sensors (CROSS), localized game theoretical clustering algorithm (LGCA), and energy‐efficient clustering algorithm based on game theory (ECAGT). Simulation experiments validate that the proposed algorithm enhances the network's lifetime up to 47.49% compared to existing clustering algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

“…34 Optimized resource scheduling strategy for speculative execution (ORSE) algorithm is proposed using the non-cooperative game strategy for scheduling of resources of straggling tasks. 35…”

Section: Related Workmentioning

confidence: 99%

A game theory‐based clustering and multi‐hop routing scheme in wireless sensor networks for energy minimization

Gupta

Aulakh

2022

Int J Communication

View full text Add to dashboard Cite

show abstract

“…That is, when the original data are entered, they are divided by the RS erasure coding (6, 3) using as many blocks of data. In addition, each data block generates parity blocks through an encoding matrix and calculations, and thus all blocks (data blocks and parity blocks) are distributed and stored for each disk or network environment they use, as shown in Equation (1).…”

Section: Principle Of Erasure Codingmentioning

confidence: 99%

“…Owing to the recent development of technologies such as smartphones, IoT, artificial intelligence, and big data, large-capacity big data are being generated and utilized. We previously used a replication technology-based distributed file system to store such data; however, with the spread of cloud computing, we have recently begun storing big data more efficiently using an erasure coding-based distributed file system [1][2][3]. Replication techniques divide the original data into multiple data blocks and store them on each distributed server through n-duplex replication.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems

Shin

Kim

2021

Applied Sciences

View full text Add to dashboard Cite

Research has been conducted to efficiently transfer blocks and reduce network costs when decoding and recovering data from an erasure coding-based distributed file system. Technologies using software-defined network (SDN) controllers can collect and more efficiently manage network data. However, the bandwidth depends dynamically on the number of data transmitted on the network, and the data transfer time is inefficient owing to the longer latency of existing routing paths when nodes and switches fail. We propose deep Q-network erasure coding (DQN-EC) to solve routing problems by converging erasure coding with DQN to learn dynamically changing network elements. Using the SDN controller, DQN-EC collects the status, number, and block size of nodes possessing stored blocks during erasure coding. The fat-tree network topology used for experimental evaluation collects elements of typical network packets, the bandwidth of the nodes and switches, and other information. The data collected undergo deep reinforcement learning to avoid node and switch failures and provide optimized routing paths by selecting switches that efficiently conduct block transfers. DQN-EC achieves a 2.5-times-faster block transmission time and 0.4-times-higher network throughput than open shortest path first (OSPF) routing algorithms. The bottleneck bandwidth and transmission link cost can be reduced, improving the recovery time approximately twofold.

show abstract

Big Data Technologies with Computational Model Computing Using Hadoop with Scheduling Challenges

Priyanka¹,

Thangavel

Meenakshipriya

et al. 2021

Studies in Computational Intelligence

View full text Add to dashboard Cite

An Optimized Resource Scheduling Strategy for Hadoop Speculative Execution Based on Non-cooperative Game Schemes

Cited by 5 publications

References 18 publications

A game theory‐based clustering and multi‐hop routing scheme in wireless sensor networks for energy minimization

A game theory‐based clustering and multi‐hop routing scheme in wireless sensor networks for energy minimization

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems

Big Data Technologies with Computational Model Computing Using Hadoop with Scheduling Challenges

Contact Info

Product

Resources

About