A Double Q-Learning Routing in Delay Tolerant Networks

Yuan, Fan; Wu, Jaogao; Zhou, Hongyu; Liu, Linfeng

doi:10.1109/icc.2019.8761526

Cited by 23 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, using Q-Learning in DTNs can also suffer a large penalty, because it can produce a positive bias by using the maximum value as the approximation of the maximum expected value. Therefore, we proposed DQLR [11] to solve the above problems. In the DQLR protocol, the Double Q -Learning algorithm is used to decouple the selection from the evaluation to obtain an unbiased estimation.…”

Section: Related Workmentioning

confidence: 99%

“…But, the DTBR protocol takes the maximum action value as the optimal action, which may be obscured by overestimation. Therefore, in our previous work, the Double Q-Learning Routing (DQLR) protocol was proposed [11], which adopts the Double Q-Learning algorithm to obtain an unbiased estimation and improve the performance of message delivery. However, in real scenarios of DTNs, the characteristics of nodes (e.g., node activity, contact interval, and movement speed) are complex, dynamic, and uncertain, which will affect the performance of routing protocols.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Fuzzy‐Logic‐Based Double Q‐Learning Routing in Delay‐Tolerant Networks

Yuan

Guo

et al. 2021

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

Delay-tolerant networks (DTNs) are wireless mobile networks, which suffer from frequent disruption, high latency, and lack of a complete path from source to destination. The intermittent connectivity in DTNs makes it difficult to efficiently deliver messages. Research results have shown that the routing protocol based on reinforcement learning can achieve a reasonable balance between routing performance and cost. However, due to the complexity, dynamics, and uncertainty of the characteristics of nodes in DTNs, providing a reliable multihop routing in DTNs is still a particular challenge. In this paper, we propose a Fuzzy-logic-based Double Q -Learning Routing (FDQLR) protocol that can learn the optimal route by combining fuzzy logic with the Double Q -Learning algorithm. In this protocol, a fuzzy dynamic reward mechanism is proposed, and it uses fuzzy logic to comprehensively evaluate the characteristics of nodes including node activity, contact interval, and movement speed. Furthermore, a hot zone drop mechanism and a drop mechanism are proposed, which can improve the efficiency of message forwarding and buffer management of the node. The simulation results show that the fuzzy logic can improve the performance of the FDQLR protocol in terms of delivery ratio, delivery delay, and overhead. In particular, compared with other related routing protocols of DTNs, the FDQLR protocol can achieve the highest delivery ratio and the lowest overhead.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Fuzzy‐Logic‐Based Double Q‐Learning Routing in Delay‐Tolerant Networks

Yuan

Guo

et al. 2021

Wireless Communications and Mobile Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The situation is quite similar to in SSA-MAC: the nano-nodes have to harvest energy and wait for their own slots, which could also cause latency and intermittent connectivity. For example, a double Q-learning routing algorithm is proposed for DTN to improve the connective efficiency, while balancing the routing performance and cost [45]. In [46], reliable energy-aware routing protocol for DTN is proposed to address the energy depletion of nodes.…”

Section: The Proposed Ssa-mac Protocolmentioning

confidence: 99%

Slot Self-Allocation Based MAC Protocol for Energy Harvesting Nano-Networks

Wang

Yao

2019

Sensors

View full text Add to dashboard Cite

Nano-networks are composed of interconnected nano-nodes and can enable unprecedented applications in various fields. Due to the peculiarities of nano-networks, such as high density, extremely limited energy and computational resources, traditional carrier-sensing based Media Access Control (MAC) protocols are not suitable for nano-networks. In this paper, a Slot Self-Allocation based MAC protocol (SSA-MAC) is proposed for energy harvesting nano-networks. Two transmission schemes for centralized and distributed nano-networks are designed, respectively. In centralized nano-networks, nano-nodes can only send packets to the nano-controller in their Self-Allocation Slots (SASs), while, in distributed nano-networks, nano-nodes can only receive packets from surrounding nano-nodes in their SASs. Extensive simulations were conducted to compare the proposed SSA-MAC with PHysical LAyer aware MAC (PHLAME), Receiver-Initiated Harvesting-aware MAC (RIH-MAC) and Energy Efficient Wireless NanoSensor Network MAC (EEWNSN). From the results, it can be concluded that the proposed SSA-MAC achieves better performance and can reduce the collision probability, while improving the energy efficiency of nano-networks.

show abstract

“…Double Q-Learning Routing(DQLR) [22], selects the next hop in a distributed way. DQLR decouples the selection and evaluation with two value functions ie., the double Q-Learning functions.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…Bayesian Classifier [8], [9], [10] K-Means Clustering [2] Principal Componenet Analysis [18] Q-Learning [13], [22], [25] [ 14] high trust value towards the destination, then that intermediate node is chosen for routing. If the intermediate nodes have the same trust value, then the routing decision is based on the latest connection time.…”

Section: A Routingmentioning

confidence: 99%

Machine Learning in Delay Tolerant Networks: Algorithms, Strategies, and Applications

2019

IJITEE

View full text Add to dashboard Cite

Delay Tolerant Networks (DTNs) has intermittent connectivity, nodes in the network experience a long delay in the delivery of packets, and the nodes are sparsely distributed. DTN is deployed in the applications where human intervention is least like underwater communication, interplanetary communication, disaster management, tracking wildlife, etc. Any changes in the environment affect the deployed sensor nodes, so it is required that the sensor nodes adapt to these environmental changes. Machine-Learning (ML) techniques can be deployed to overcome such difficulty. ML improves the network lifetime. ML in DTN facilitates routing by adapting to the network changes, mitigates congestion, reduces overhead. This paper provides a survey of ML techniques used in DTN. To the best of our knowledge, this work is the first of its kind to survey ML techniques in DTN

show abstract

A Double Q-Learning Routing in Delay Tolerant Networks

Cited by 23 publications

References 18 publications

A Fuzzy‐Logic‐Based Double Q‐Learning Routing in Delay‐Tolerant Networks

A Fuzzy‐Logic‐Based Double Q‐Learning Routing in Delay‐Tolerant Networks

Slot Self-Allocation Based MAC Protocol for Energy Harvesting Nano-Networks

Machine Learning in Delay Tolerant Networks: Algorithms, Strategies, and Applications

Contact Info

Product

Resources

About