Resource coordination in wireless sensor networks by cooperative reinforcement learning

Khan, Muhidul Islam; Rinner, Bernhard

doi:10.1109/percomw.2012.6197639

Cited by 20 publications

(30 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Khan and Rinner [8] apply reinforcement learning (RL) for online task scheduling. They use cooperative reinforcement learning for task scheduling.…”

Section: Reinforcement Learning-based Methodsmentioning

confidence: 99%

“…Khan and Rinner [8] propose cooperative Q learning (RL) where every agent needs to maintain a Q matrix for the value functions like independent Q learning. Initially, all entries of the Q matrix are 0 and the nodes or agents may be in any state.…”

Section: Rlmentioning

confidence: 99%

“…The balancing factor of the reward function is varied, and the number of nodes in the network and the randomness of moving targets to find out the tracking quality/energy consumption trade-off are also varied. The average execution time and communication overhead for distributed independent reinforcement learning (DIRL) [7], reinforcement learning (RL) [8], cooperative reinforcement learning (CRL) [9], and Exp3 are also calculated.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

Khan

2016

J Wireless Com Network

Self Cite

View full text Add to dashboard Cite

A wireless sensor network (WSN) is composed of a large number of tiny sensor nodes. Sensor nodes are very resource-constrained, since nodes are often battery-operated and energy is a scarce resource. In this paper, a resource-aware task scheduling (RATS) method is proposed with better performance/resource consumption trade-off in a WSN. Particularly, RATS exploits an adversarial bandit solver method called exponential weight for exploration and exploitation (Exp3) for target tracking application of WSN. The proposed RATS method is compared and evaluated with the existing scheduling methods exploiting online learning: distributed independent reinforcement learning (DIRL), reinforcement learning (RL), and cooperative reinforcement learning (CRL), in terms of the tracking quality/energy consumption trade-off in a target tracking application. The communication overhead and computational effort of these methods are also computed. Simulation results show that the proposed RATS outperforms the existing methods DIRL and RL in terms of achieved tracking performance.

show abstract

“…Khan and Rinner [8] apply reinforcement learning (RL) for online task scheduling. They use cooperative reinforcement learning for task scheduling.…”

Section: Reinforcement Learning-based Methodsmentioning

confidence: 99%

Section: Rlmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

Khan

2016

J Wireless Com Network

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [3], the problem of finding an efficient sleep-wake policy for the sensors while maintaining good tracking accuracy by solving an MDP has been studied. In [20], the authors propose a Q-learning based algorithm for sleep scheduling. In [14,15], the authors propose a POMDP model for sleepscheduling in an object tracking application and propose several algorithms based on traditional dynamic programming approaches to solve this problem.…”

Section: Related Workmentioning

confidence: 99%

“…Update θ n using (20) Slow timescale θ n+1 g(s n , a n ) Fig. 3 Overall flow of the TQSA-A algorithm from the perturbed simulation, the gradient of the approximate Q-value function Qðs; aÞ % h T r s;a is estimated as:…”

Section: Fast Timescalementioning

confidence: 99%

Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

2014

View full text Add to dashboard Cite

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), [2091][2092][2093][2094][2095][2096][2097][2098][2099][2100][2101] 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

show abstract

Application of reinforcement learning to wireless sensor networks: models and algorithms

et al. 2014

View full text Add to dashboard Cite

Wireless sensor network (WSN) consists of a large number of sensors and sink nodes which are used to monitor events or environmental parameters, such as movement, temperature, humidity, etc. Reinforcement learning (RL) has been applied in a wide range of schemes in WSNs, such as cooperative communication, routing and rate control, so that the sensors and sink nodes are able to observe and carry out optimal actions on their respective operating environment for network and application performance enhancements. This article provides an extensive review on the application of RL to WSNs. This covers many components and features of RL, such as state, action and reward. This article presents how most schemes in WSNs have been approached using the traditional and enhanced RL models and algorithms. It also presents performance enhancements brought about by the RL algorithms, and open issues associated with the application of RL in WSNs. This article aims to establish a foundation in order to spark new research interests in this area. Our discussion has been presented

show abstract

Resource coordination in wireless sensor networks by cooperative reinforcement learning

Cited by 20 publications

References 7 publications

Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

Resource-aware task scheduling by an adversarial bandit solver method in wireless sensor networks

Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

Application of reinforcement learning to wireless sensor networks: models and algorithms

Contact Info

Product

Resources

About