Channel Selection for Network-Assisted D2D Communication via No-Regret Bandit Learning With Calibrated Forecasting

Abstract

We consider the distributed channel selection problem in the context of device-to-device (D2D) communication as an underlay to a cellular network. Underlaid D2D users communicate directly by utilizing the cellular spectrum, but their decisions are not governed by any centralized controller. Selfish D2D users that compete for access to the resources form a distributed system where the transmission performance depends on channel availability and quality. This information, however, is difficult to acquire. Moreover, the adverse effects of D2D users on cellular transmissions should be minimized. In order to overcome these limitations, we propose a network-assisted distributed channel selection approach in which D2D users are only allowed to use vacant cellular channels. This scenario is modeled as a multi-player multi-armed bandit game with side information, for which a distributed algorithmic solution is proposed. The solution is a combination of no-regret learning and calibrated forecasting, and can be applied to a broad class of multi-player stochastic learning problems, in addition to the formulated channel selection problem. Theoretical analysis shows that the proposed approach not only yields vanishing regret in comparison to the global optimal solution but also guarantees that the empirical joint frequencies of the game converge to the set of correlated equilibria.

Machine Learning Paradigms for Next-Generation Wireless Networks

“…Generally, these models may be beneficially used in multi-player adaptive decision making problems, where selfish players infer an optimal joint action profile from their successive interactions with a dynamic environment, and finally settle at some equilibrium point. This problem has indeed been encountered in many wireless networking scenarios, with a compelling one being the channel selection problem of a distributed device-to-device (D2D) communication system integrated into a cellular network, and another one in the context of emerging next-generation networks [16]. The selfish D2D users aimed to optimize their own performance by camping on the vacant cellular channels, whose statistics were unknown to the users.…”

“…• Exploration vs. exploitation • Multi-armed bandit game Device-to-device networks [16] ciation under the unknown energy status of the base stations in energy harvesting networks.…”

A Bandit Approach to Price-Aware Energy Management in Cellular Networks

“…Similar to [7], the proposed algorithm enables smart scheduling that linearly increases the number of exploration with an exponentially increased number of time slots, as presented in Fig. 1 and Table 1, which reduces the exploration overhead in terms of total energy cost over a finite time horizon.…”

An Autonomous Learning-Based Algorithm for Joint Channel and Power Level Selection by D2D Pairs in Heterogeneous Cellular Networks

“…The majority of existing approaches in this area (e.g., [2] - [8]) consider the network as a single entity, where the resources are allocated by some BS that has a global knowledge of the precise or statistical channel state information (CSI). However, since conventional pilot signals (deployed for cellular communication) cannot be used for estimation of the D2D channels [9], the assumption of the precise CSI in a D2D-enabled cellular network is somewhat unrealistic. As a result, many researchers have argued in favor of an alternative resource allocation strategy, where the D2D users establish a secondary network that is allowed to occupy the vacant cellular bands, thereby causing no interference to primary (cellular) users.…”

“…Besides, the stability of a final solution was not verified. In [9], the opportunistic D2D access has been formulated as a multi-agent learning game, where the players (D2D users) learn the optimal action from successive interactions with a dynamic environment (orthogonal cellular channel). Each D2D user is selfish and aims at optimizing its throughput performance while being allowed to transmit on the vacant channels using a carrier sense multiple access with collision avoidance (CSMA/CA).…”