Abstract-Coexistence of multiple radio access technologies (RATs) is a promising paradigm to improve spectral efficiency. This letter presents a game-theoretic network selection scheme in a cognitive heterogeneous networking environment with timevarying channel availability. We formulate the network selection problem as a noncooperative game with secondary users (SUs) as the players, and show that the game is an ordinal potential game (OPG). A decentralized, stochastic learning-based algorithm is proposed where each SU's strategy progressively evolves toward the Nash equilibrium (NE) based on its own actionreward history, without the need to know actions in other SUs. The convergence properties of the proposed algorithm toward an NE point are theoretically and numerically verified. The proposed algorithm demonstrates good throughput and fairness performances in various network scenarios.

Abstract-Performance analysis of multiuser orthogonal frequency division multiplexing (OFDM-TDMA) and orthogonal frequency division multiple access (OFDMA) networks in support of multimedia transmission is conducted in this work. We take a cross-layer approach and analyze several quality-of-service (QoS) measures that include the bit rate and the bit error rate (BER) in the physical layer, and packet average throughput/delay and packet maximum delay in the link layer. We adopt a cross-layer QoS framework similar to that in IEEE 802.16, where service classification, flow control and opportunistic scheduling with different subcarrier/bit allocation schemes are implemented. In the analysis, the Rayleigh fading channel in the link layer is modeled by a finite-state Markov chain, and the channel state information (CSI) is assumed to be available at the base station. With the M/G/1 queueing model and flow control results, our analysis provides important insights into the performance difference of these two multiaccess systems. The derived analytical results are verified by extensive computer simulation. It is demonstrated by analysis and simulation that OFDMA outperforms OFDM-TDMA in QoS metrics of interest. Thus, OFDMA has higher potential than OFDM-TDMA in supporting multimedia services.Index Terms-quality of services (QoS), multiple access, OFDM, OFDMA, cross-layer analysis, opportunistic scheduling.

No abstract

No abstract

This paper studies the problem of joint power allocation and user association in wireless heterogeneous networks (HetNets) with a deep reinforcement learning (DRL)-based approach. This is a challenging problem since the action space is hybrid, consisting of continuous actions (power allocation) and discrete actions (device association). Instead of quantizing the continuous space (i.e., possible values of powers) into a set of discrete alternatives and applying traditional deep reinforcement approaches such as deep Q learning, we propose working on the hybrid space directly by using the novel parameterized deep Q-network (P-DQN) to update the learning policy and maximize the average cumulative reward. Furthermore, we incorporate the constraints of limited wireless backhaul capacity and the quality-of-service (QoS) of each user equipment (UE) into the learning process. Simulation results show that the proposed P-DQN outperforms the traditional approaches, such as the DQN and distance-based association, in terms of energy efficiency while satisfying the QoS and backhaul capacity constraints. The improvement in the energy efficiency of the proposed P-DQN on average may reach 77.6% and 140.6% over the traditional DQN and distance-based association approaches, respectively, in a HetNet with three SBS and five UEs.