Learning with handoff cost constraint for network selection in heterogeneous wireless networks

Abstract

In heterogeneous wireless networks, network selection algorithms provide the user with the optimum network access choice. The optimal network is evaluated according to network parameters. Considering that the network parameters are dynamic and unavailable for the user in realistic heterogeneous wireless network environments, most existing network selection algorithms cannot work effectively. Learning‐based algorithms can address the problem of uncertain network parameters, while they commonly need considerable network handoff, resulting in unbearable handoff cost. In order to tackle the uncertainty of network parameters, we formulate the network selection problem as a multi‐armed bandit problem. Moreover, two online learning‐based network selection algorithms with a special consideration on reducing network handoff cost are proposed. By updating in a block manner, both algorithms achieve optimal logarithmic‐order regret and limited network handoff cost. The simulation indicates that the two algorithms can significantly reduce the network handoff cost and improve the transmission performance compared with existing algorithms, simultaneously. Copyright © 2014 John Wiley & Sons, Ltd.

Biased Stackelberg game‐based UAV relay anti‐jamming communications: Exploiting trajectory optimization and transmission mode selection

“…Similar to refs. [41,56], we can obtain the upper bound of the expected regret by using the given equations…”

“…Similar to refs. [41, 56], we can obtain the upper bound of the expected regret by using the given equations $$\begin{flalign} E{\left[ {R{\left(T \right)}} \right]} & \le \sum \limits _{{q: \: {{\widehat{U}^{\prime }}}_q < {{\widehat{U}^{\prime }}}_{{q^ * }}}} {{\left({{{\widehat{U}^{\prime }}}_{{q^ * }} - {{\widehat{U}^{\prime }}}_q} \right)}} E{\left[ {num{\left(T \right)}} \right]} \nonumber \\ & \le \sum \limits _{{q: \: {{\widehat{U}^{\prime }}}_q < {{\widehat{U}^{\prime }}}_{{q^ * }}}} {{\Delta _q}{\left({\frac{{8\ln S{\left(T \right)}}}{{\Delta _q^2}} + 1 + \frac{{{\pi ^2}}}{3}} \right)}} \nonumber \\ & \le \sum \limits _{{q: \: {{\widehat{U}^{\prime }}}_q < {{\widehat{U}^{\prime }}}_{{q^ * }}}} {{\Delta _q}{\left({\frac{{8\ln T}}{{\Delta _q^2}} + 1 + \frac{{{\pi ^2}}}{3}} \right)}}. \end{flalign}$$…”

Shrewd Selection Speeds Surfing: Use Smart EXP3!

“…This ensures that more time is spent in the optimal network, which is eventually selected more frequently. The use of blocks reduces switching cost [3], [13], [23] and improves performance by de-synchronizing the selection time of devices. Every so often and upon significant decline in network quality, block lengths are reset for better adaptation.…”

“…But the adversarial bandit problem, where an adversary determines the payoff for each arm, can be easily related to a repeated multi-player game [4]. While EXP3 [4] ignores switching cost, the concept of updating in a block manner has been proposed [3], [13], [23] to take into account switching cost. Multi-armed bandit techniques have also been applied to other resource selection problems, such as channel selection [17], [34], selection of the appropriate sensors to query in a sensor network [19], and selection of replica server for content distribution networks [35].…”

Group vertical handoff management in heterogeneous networks

“…In case of group mobility (e.g., a group of passengers equipped with mobile terminals on board a bus or train) whereby a group of mobile terminals enter into the service area of a heterogeneous network, the selection (by each mobile terminal) of a suitable RAT is a crucial decision. Such RAT selection is not supposed to be done in a random way, but rather on the basis of certain criteria such as radio signal strength (RSS), quality of service (QoS) parameters , quality of experience (QoE) criteria , individual consideration and contextual information , handoff cost in the uncertainty of QoS parameters , and signal to interference plus noise ratio (SINR) parameter . However, most state of the art RAT selection mechanisms and schemes do not consider the impact introduced by decisions of other mobile terminals.…”