Jointly optimized bit-rate/delay control policy for wireless packet networks with fading channels

Razavilar, J.; Liu, K.J.R.; Marcus, Solomon

doi:10.1109/26.990910

Cited by 98 publications

(44 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Note that in the previous model, the transmitter acquires this delayed CSI only if the S or T h actions are taken 5 . But, if the action T l is taken instead, the CSI is not known since in all cases the transmission is successful.…”

Section: A Structure Of the Value Function And Of The Optimal Policymentioning

confidence: 99%

“…One can also view β as the probability that a particular user is allowed to use the channel (see [5] for further details). Finally, from Th.6.17 and Th.6.18 in [19], it can be seen also that the discounted reward criterion is of primary importance when it comes to the derivation for the optimal policy of the average reward criterion (throughput).…”

Section: Mdp Formulationmentioning

confidence: 99%

“…In [4], the authors considered rate and power control strategies for transmitting a fixed number of bits over fading channels subject to both energy and delay constraints. In [5], the authors obtained the optimal rate control policy in wireless networks with Rayleigh fading channels. Most related to this paper are [6] and [7].…”

Section: A Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Betting on Gilbert-Elliot channels

Laourine

Tong

2010

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

In this paper a communication system operating over a Gilbert-Elliot channel is studied. The goal of the transmitter is to maximize the number of successfully transmitted bits. This is achieved by choosing among three possible actions: (i) betting aggressively by using a weak code that allows the transmission of a high number of bits but provides no protection against a bad channel, ii) betting conservatively by using a strong code that perfectly protects the transmitted bits against a bad channel but does not allow a high number of data bits, iii) betting opportunistically by sensing the channel for a fixed duration and then deciding which code to use. The problem is formulated and solved using the theory of Markov decision processes (MDPs). It is shown that the optimal strategy has a simple threshold structure. Closed form expressions and simplified procedures for the computation of the threshold policies in terms of the system parameters are provided. If a feedback channel is available, it is shown that the total number of transmitted bits increases.

show abstract

Section: A Structure Of the Value Function And Of The Optimal Policymentioning

confidence: 99%

Section: Mdp Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Betting on Gilbert-Elliot channels

Laourine

Tong

2010

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

show abstract

“…1. Assuming slow fading conditions, the state transition probability of the MCS level during the frame duration T f is given by (Liu et al, 2005;Razavilar et al, 2002) …”

Section: Channel Modelmentioning

confidence: 99%

Scheduling and Capacity of VoIP Services in Wireless OFDMA Systems

So¹

2011

VoIP Technologies

View full text Add to dashboard Cite

“…They show that the DP solution is better than the fixed signal-to-interference ratio (SIR) approach. Jointly optimized bit-rate and delay control for packet wireless networks has also been studied within the DP framework [8]. Most of the literatures assume the knowledge of the exact probability model and obtain the optimal solution by solving Bellman's optimality equation [9].…”

mentioning

confidence: 99%

Near-optimal reinforcement learning framework for energy-aware sensor communications

Pandana

Liu

2005

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

Abstract-We consider the problem of average throughput maximization per total consumed energy in packetized sensor communications. Our study results in a near-optimal transmission strategy that chooses the optimal modulation level and transmit power while adapting to the incoming traffic rate, buffer condition, and the channel condition. We investigate the point-to-point and multinode communication scenarios. Many solutions of the previous works require the state transition probability, which may be hard to obtain in a practical situation. Therefore, we are motivated to propose and utilize a class of learning algorithms [called reinforcement learning (RL)] to obtain the near-optimal policy in point-to-point communication and a good transmission strategy in multinode scenario. For comparison purpose, we develop the stochastic models to obtain the optimal strategy in the point-to-point communication. We show that the learned policy is close to the optimal policy. We further extend the algorithm to solve the optimization problem in a multinode scenario by independent learning. We compare the learned policy to a simple policy, where the agent chooses the highest possible modulation and selects the transmit power that achieves a predefined signal-to-interference ratio (SIR) given one particular modulation. The proposed learning algorithm achieves more than twice the throughput per energy compared with the simple policy, particularly, in high packet arrival regime. Beside the good performance, the RL algorithm results in a simple, systematic, self-organized, and distributed way to decide the transmission strategy.Index Terms-Energy-aware sensor communications, Markov decision process (MDP), reinforcement learning (RL).

show abstract

Jointly optimized bit-rate/delay control policy for wireless packet networks with fading channels

Cited by 98 publications

References 16 publications

Betting on Gilbert-Elliot channels

Betting on Gilbert-Elliot channels

Scheduling and Capacity of VoIP Services in Wireless OFDMA Systems

Near-optimal reinforcement learning framework for energy-aware sensor communications

Contact Info

Product

Resources

About