Multiobjective Load Balancing for Multiband Downlink Cellular Networks: A Meta- Reinforcement Learning Approach

Feriani, Amal; Wu, Di; Xu, Yi; Li, Jimmy; Jang, Seowoo; Hossain, Ekram; Li, Xue; Dudek, Gregory

doi:10.1109/jsac.2022.3191114

Cited by 19 publications

(10 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For any ∈ (0, 1), a solution θ is found such that E[||θ ||] 2 ≤ + 2 , where is the meta training parameters. The convergence of multi-step MAML for RL is guaranteed to converge in O( 2 ) iterations if meta learning rate is chosen properly (Feriani et al, 2022). Therefore, the MAML-RL is basically guaranteed to converge.…”

Section: Appendix Bmentioning

confidence: 99%

Energy-efficient tool path generation and expansion optimisation for five-axis flank milling with meta-reinforcement learning

Lu,

Zhou,

Zhang

et al. 2024

J Intell Manuf

View full text Add to dashboard Cite

Five-axis flank milling is prevalent in complex surfaces manufacturing, and it typically consumes high electricity energy. To save energy and improve energy efficiency, this paper proposes a tool path optimisation of five-axis flank milling by meta-reinforcement learning. Firstly, considering flank milling features, a feed angle is defined that guides tool spatial motion and identifies an ideal principal path. Then, machining energy consumption and time are modelled by tool path variables, i.e., feed angle, cutting strip width and path length. Secondly, an energy-efficient tool path dynamic optimisation model is constructed, which is then described by multiple Markov Decision Processes (MDPs). Thirdly, meta-learning integrating with the Soft Actor-Critic (MSAC) framework is utilised to address the MDPs. In an MDP with one principal path randomly generated by a feed angle, cutting strip width is dynamically optimised under a maximum scallop height limit to realise energy-efficient multi-expansions. By quick traversal of MDPs with various feed angles, MSAC enables an energy-efficient path generation and expansion integrated scheme. Experiments show that, regarding machining energy consumption and time, the proposed method achieves a reduction of 69.96% and 68.44% over the end milling with an iso-scallop height, and of 41.50% and 39.80% over the flank milling with an iso-scallop height, with a minimum amount of machining carbon emission, which highlights its contribution to the arena of energy-oriented and sustainable intelligent manufacturing.

show abstract

Section: Appendix Bmentioning

confidence: 99%

Energy-efficient tool path generation and expansion optimisation for five-axis flank milling with meta-reinforcement learning

Lu,

Zhou,

Zhang

et al. 2024

J Intell Manuf

View full text Add to dashboard Cite

show abstract

“…Frequent handovers can lead to increased connection drop rates, higher transmission delays, and induce ping-pong effects [4]. These can be especially detrimental for heterogeneous deployments which typically have a smaller average cell size.…”

Section: A User Service Ratementioning

confidence: 99%

“…As the industry trends towards intelligent and open networks [21], [22], network measurements across BSs and users can be leveraged to make data-driven user association and load balancing decisions. As such, mobility-aware user association has been considered recently as a reinforcement learning (RL) problem [4], [23]- [30]. Among these, [26], [28] aim to optimize the network utility within the RL framework, [23], [24] maximize the sum-rate, and [4], [27] optimize cell individual offset parameters.…”

mentioning

confidence: 99%

“…As such, mobility-aware user association has been considered recently as a reinforcement learning (RL) problem [4], [23]- [30]. Among these, [26], [28] aim to optimize the network utility within the RL framework, [23], [24] maximize the sum-rate, and [4], [27] optimize cell individual offset parameters. However, such model-free RL techniques are known to be sample-inefficient [31], and there has been a push towards scalable and sample-efficient formulations [29], [32], [33].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Forecaster-aided User Association and Load Balancing in Multi-band Mobile Networks

Gupta¹,

Chinchali²,

Varkey³

et al. 2023

Preprint

View full text Add to dashboard Cite

Cellular networks are becoming increasingly heterogeneous with higher base station (BS) densities and ever more frequency bands, making BS selection and band assignment key decisions in terms of rate and coverage. In this paper, we decompose the mobility-aware user association task into (i) forecasting of user rate and then (ii) convex utility maximization for user association accounting for the effects of BS load and handover overheads. Using a linear combination of normalized mean-squared error (NMSE) and normalized discounted cumulative gain (NDCG) as a novel loss function, a recurrent deep neural network is trained to reliably forecast the mobile users' future rates. Based on the forecast, the controller optimizes the association decisions to maximize the service rate-based network utility using our computationally efficient (speed up of 100× versus generic convex solver) algorithm based on the Frank-Wolfe method. Using an industry-grade network simulator developed by Meta, we show that the proposed model predictive control (MPC) approach improves the 5th percentile service rate by 3.5× compared to the traditional signal strength-based association, reduces the median number of handovers by 7× compared to a handover agnostic strategy, and achieves service rates close to a genie-aided scheme. Furthermore, our model-based approach is significantly more sample-efficient (needs 100× less training data) compared to model-free reinforcement learning (RL), and generalizes well across different user drop scenarios. I. INTRODUCTIONCellular networks continue to trend towards multi-tier and multi-band deployments to meet intense consumer demands for faster and more ubiquitous data connectivity. The use of low-power small cell technology enables flexible deployments and helps boost network capacity through increased spatial reuse. Meanwhile, more spectrum is being released, for instance almost 500 M. Gupta, S. Chinchali, and J. G. Andrews are with 6G@UT in the Wireless Networking and Communications Group at the

show abstract

“…It is also possible to use combinations of different events to drive the load balancing process. For example, Feriani et al (2022) considers a combination of A2 and A5 events. Specifically, if the following two conditions are satisfied, then a handover is triggered.…”

Section: Mlb Via Handover For Active Uesmentioning

confidence: 99%

Reinforcement learning for communication load balancing: approaches and challenges

Wu,

Li,

Ferini

et al. 2023

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

The amount of cellular communication network traffic has increased dramatically in recent years, and this increase has led to a demand for enhanced network performance. Communication load balancing aims to balance the load across available network resources and thus improve the quality of service for network users. Most existing load balancing algorithms are manually designed and tuned rule-based methods where near-optimality is almost impossible to achieve. Furthermore, rule-based methods are difficult to adapt to quickly changing traffic patterns in real-world environments. Reinforcement learning (RL) algorithms, especially deep reinforcement learning algorithms, have achieved impressive successes in many application domains and offer the potential of good adaptabiity to dynamic changes in network load patterns. This survey presents a systematic overview of RL-based communication load-balancing methods and discusses related challenges and opportunities. We first provide an introduction to the load balancing problem and to RL from fundamental concepts to advanced models. Then, we review RL approaches that address emerging communication load balancing issues important to next generation networks, including 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions for applying RL for communication load balancing.

show abstract

Multiobjective Load Balancing for Multiband Downlink Cellular Networks: A Meta- Reinforcement Learning Approach

Cited by 19 publications

References 36 publications

Energy-efficient tool path generation and expansion optimisation for five-axis flank milling with meta-reinforcement learning

Energy-efficient tool path generation and expansion optimisation for five-axis flank milling with meta-reinforcement learning

Forecaster-aided User Association and Load Balancing in Multi-band Mobile Networks

Reinforcement learning for communication load balancing: approaches and challenges

Contact Info

Product

Resources

About