Learning-NUM: Network Utility Maximization With Unknown Utility Functions and Queueing Delay

Abstract

Network Utility Maximization (NUM) studies the problems of allocating traffic rates to network users in order to maximize the users' total utility subject to network resource constraints. In this paper, we propose a new NUM framework, Learning-NUM, where the users' utility functions are unknown apriori and the utility function values of the traffic rates can be observed only after the corresponding traffic is delivered to the destination, which means that the utility feedback experiences queueing delay. The goal is to design a policy that gradually learns the utility functions and makes rate allocation and network scheduling/routing decisions so as to maximize the total utility obtained over a finite time horizon 𝑇 . In addition to unknown utility functions and stochastic constraints, a central challenge of our problem lies in the queueing delay of the observations, which may be unbounded and depends on the decisions of the policy. We first show that the expected total utility obtained by the best dynamic policy is upper bounded by the solution to a static optimization problem. Without the presence of feedback delay, we design an algorithm based on the ideas of gradient estimation and Max-Weight scheduling. To handle the feedback delay, we embed the algorithm in a parallel-instance paradigm to form a policy that achieves Õ (𝑇 3/4 )-regret, i.e., the difference between the expected utility obtained by the best dynamic policy and our policy is in Õ (𝑇 3/4 ). Finally, to demonstrate the practical applicability of the Learning-NUM framework, we apply it to three application scenarios including database query, job scheduling and video streaming. We further conduct simulations on the job scheduling application to evaluate the empirical performance of our policy.

Network Utility Maximization with Unknown Utility Functions: A Distributed, Data-Driven Bilevel Optimization Approach

“…In this section, we consider a real-world network, Abilene network, whose topology is shown in Figure 4. Following the setup in [14], this network contains four data transmission flows with distinct underlying utilities, where flow 1 has a quadratic utility a 1 x 2 , flow 2 has a square root utility…”

“…Network utility maximization (NUM) has been studied for decades since the seminal work [1] and has been the central analytical framework for the design of fair and distributed resource allocation over the communication networks (e.g., the Internet, 6G networks). Its applications span from network congestion control [1,2,3], power allocation and routing in wireless networks [4,5], load scheduling in cloud computing [6,7,8], to video streaming over dynamic networks [9,10,11,12,13,14], and etc. A comprehensive introduction of the method and its connections to control theory and convex optimization can be found in [15].…”

“…The solutions maximize the total network utility subject to resource constraints such as channel capacity, average power, etc. There have been a large body of studies on NUM for wired [1,2,3,15,16,17,18,19] and wireless networks [20,21,22,23,24,25,26,11,12,13,27,14]. These existing studies on NUM almost exclusively assume that the utility functions are known to the users and are concave, e.g.…”

“…The answer to this question has not been explored well except a few recent attempts [29,14] using online learning algorithms. For example, [14] focused on a stochastic dynamic scenario, and proposed an online policy to gradually learn the utility functions and allocate resources accordingly. However, it still assumes the unknown utility functions are concave and requires a central scheduler.…”

Reinforcement Learning-based Wi-Fi Contention Window Optimization

“…In [19], the authors present a Deep Q-learning algorithm to dynamically adapt CWmin to random access in wireless networks. The idea is to maximize a network utility function (i.e., a metric measuring the fair use of the medium) [20] under dynamic and uncertain scenarios by rewarding the actions that lead to high utilities (efficient resource usage). The proposed solution employs an intelligent node, called node 0, that implements the DQN algorithm to choose the CWmin for the next time step from historical observations.…”