Structured optimal transmission control in network-coded two-way relay channels

Abstract

This paper considers a transmission control problem in network-coded two-way relay channels (NC-TWRC), where the relay buffers randomly arrived packets from two users, and the channels are assumed to be fading. The problem is modeled by a discounted infinite horizon Markov decision process (MDP). The objective is to find an adaptive transmission control policy that minimizes the packet delay, buffer overflow, transmission power consumption and downlink error rate simultaneously and in the long run. By using the concepts of submodularity, multimodularity and L -convexity, we study the structure of the optimal policy searched by dynamic programming (DP) algorithm. We show that the optimal transmission policy is nondecreasing in queue occupancies and/or channel states under certain conditions such as the chosen values of parameters in the MDP model, channel modeling method, and the preservation of stochastic dominance in the transitions of system states. Based one these results, we propose to use two low-complexity algorithms for searching the optimal monotonic policy: monotonic policy iteration (MPI) and discrete simultaneous perturbation stochastic approximation (DSPSA). We show that MPI reduces the time complexity of DP, and DSPSA is able to adaptively track the optimal policy when the statistics of the packet arrival processes change with time.

Discrete Convexity and Stochastic Approximation for Cross-layer On– off Transmission Control

“…Submodularity is a commonly seen property of queue departure controlled problems. One can refer to [5], [13]- [15] for the proofs of submodularity of Q(x, a) when different definitions of c q and c tr are used, e.g., c q = b E[f ] as in [13]. Based on Proposition 3.4, we can prove the monotonicity of the optimal policy in queue state as follows.…”