Sequential Learning for Multi-Channel Wireless Network Monitoring With Channel Switching Costs

Le, Thanh Van; Szepesvári, Csaba; Zheng, Rong

doi:10.1109/tsp.2014.2357779

Cited by 76 publications

(12 citation statements)

References 32 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…µ k a (x)}. 7 Then, arm a is defined to be lexicographically optimal for context x if there is no other arm that lexicographically dominates it in d r objectives.…”

Section: B Lexicographic Optimality For D R > 2 Objectivesmentioning

confidence: 99%

“…With the rapid increase in the generation speed of the streaming data, online learning methods are becoming increasingly valuable for sequential decision making problems. Many of these problems, including recommender systems [2], [3], medical screening [4], cognitive radio networks [5], [6] and wireless network monitoring [7] may involve multiple and possibly conflicting objectives. In this work, we propose a multi-objective contextual MAB problem with dominant and non-dominant objectives.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-objective Contextual Multi-armed Bandit With a Dominant Objective

Tekin

Turğay

2018

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

In this paper, we propose a new multi-objective contextual multi-armed bandit (MAB) problem with two objectives, where one of the objectives dominates the other objective. Unlike single-objective MAB problems in which the learner obtains a random scalar reward for each arm it selects, in the proposed problem, the learner obtains a random reward vector, where each component of the reward vector corresponds to one of the objectives and the distribution of the reward depends on the context that is provided to the learner at the beginning of each round. We call this problem contextual multi-armed bandit with a dominant objective (CMAB-DO). In CMAB-DO, the goal of the learner is to maximize its total reward in the non-dominant objective while ensuring that it maximizes its total reward in the dominant objective. In this case, the optimal arm given a context is the one that maximizes the expected reward in the non-dominant objective among all arms that maximize the expected reward in the dominant objective. First, we show that the optimal arm lies in the Pareto front. Then, we propose the multi-objective contextual multi-armed bandit algorithm (MOC-MAB), and define two performance measures: the 2-dimensional (2D) regret and the Pareto regret. We show that both the 2D regret and the Pareto regret of MOC-MAB are sublinear in the number of rounds. We also compare the performance of the proposed algorithm with other state-of-the-art methods in synthetic and real-world datasets. The proposed model and the algorithm have a wide range of real-world applications that involve multiple and possibly conflicting objectives ranging from wireless communication to medical diagnosis and recommender systems.

show abstract

“…µ k a (x)}. 7 Then, arm a is defined to be lexicographically optimal for context x if there is no other arm that lexicographically dominates it in d r objectives.…”

Section: B Lexicographic Optimality For D R > 2 Objectivesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-objective Contextual Multi-armed Bandit With a Dominant Objective

Tekin

Turğay

2018

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…The closest works to ours were presented in [32][33][34][35][36]. In [32], Arora and Szepesvari first modeled the spectrum monitoring problem as a multi-armed bandit problem (MAB) to monitor the maximum number of active users.…”

Section: A Spectrum Monitoring Problemmentioning

confidence: 99%

“…They proposed a centralized online approximation algorithm and show that it incurs sub-linear regret bounds over time and a distributed algorithm with moderate message complexity. In [34], Le et al considered switching costs for the first time and utilized Upper Confident Bound-based (UCB) policy [37] which enjoys a logarithmic regret bound in time that depends sublinearly on the number of arms, while its total switching cost grows in the order of O(log(log T)). Considering a different objective, Yi et al [35] used UCB to capture as much as interested user data.…”

Section: A Spectrum Monitoring Problemmentioning

confidence: 99%

SpecWatch: A framework for adversarial spectrum monitoring with unknown statistics

Yang

Lin

et al. 2018

Computer Networks

View full text Add to dashboard Cite

In cognitive radio networks (CRNs), dynamic spectrum access has been proposed to improve the spectrum utilization, but it also generates spectrum misuse problems. One common solution to these problems is to deploy monitors to detect misbehaviors on certain channel. However, in multi-channel CRNs, it is very costly to deploy monitors on every channel. With a limited number of monitors, we have to decide which channels to monitor. In addition, we need to determine how long to monitor each channel and in which order to monitor, because switching channels incurs costs. Moreover, the information about the misuse behavior is not available a priori. To answer those questions, we model the spectrum monitoring problem as an adversarial multi-armed bandit problem with switching costs (MAB-SC), propose an effective framework, and design two online algorithms, SpecWatch-II and SpecWatch-III, based on the same framework. To evaluate the algorithms, we use weak regret, i.e., the performance difference between the solution of our algorithm and optimal (fixed) solution in hindsight, as the metric.We prove that the expected weak regret of SpecWatch-II is O(T 2/3 ), where T is the time horizon.Whereas, the actual weak regret of SpecWatch-III is O(T 2/3 ) with probability 1 − δ, for any δ ∈ (0, 1).Both algorithms guarantee the upper bounds matching the lower bound of the general adversarial MAB-SC problem. Therefore, they are all asymptotically optimal. Index TermsCognitive Radio Networks, Spectrum Monitoring, Multi-armed Bandit Problem. This is an extended and enhanced version of the paper [50] that appeared in INFOCOM 2016.

show abstract

“…The World Wide Web technique is used here; and some communication protocols such as the TCP/IP protocol or the UDP protocol are also considered. The network sniffer and monitoring technique will be used [21] to change the information flow in real time. The proposed software system obeys the software engineering development standard, thus the inner software fault can be avoided effectively.…”

Section: Design Of Software Systemmentioning

confidence: 99%

Modeling and Simulation of Scheduling Medical Materials Using Graph Model for Complex Rescue

Lv¹,

Zheng²,

Tong³

et al. 2017

J Inf Process Syst

View full text Add to dashboard Cite

A new medical materials scheduling system and its modeling method for the complex rescue are presented. Different from other similar system, first both the BeiDou Satellite Communication System (BSCS) and the Special Fiber-optic Communication Network (SFCN) are used to collect the rescue requirements and the location information of disaster areas. Then all these messages will be displayed in a special medical software terminal. After that the bipartite graph models are utilized to compute the optimal scheduling of medical materials. Finally, all these results will be transmitted back by the BSCS and the SFCN again to implement a fast guidance of medical rescue. The sole drug scheduling issue, the multiple drugs scheduling issue, and the backup-scheme selection issue are all utilized: the Kuhn-Munkres algorithm is used to realize the optimal matching of sole drug scheduling issue, the spectral clustering-based method is employed to calculate the optimal distribution of multiple drugs scheduling issue, and the similarity metric of neighboring matrix is utilized to realize the estimation of backup-scheme selection issue of medical materials. Many simulation analysis experiments and applications have proved the correctness of proposed technique and system.

show abstract

Sequential Learning for Multi-Channel Wireless Network Monitoring With Channel Switching Costs

Cited by 76 publications

References 32 publications

Multi-objective Contextual Multi-armed Bandit With a Dominant Objective

Multi-objective Contextual Multi-armed Bandit With a Dominant Objective

SpecWatch: A framework for adversarial spectrum monitoring with unknown statistics

Modeling and Simulation of Scheduling Medical Materials Using Graph Model for Complex Rescue

Contact Info

Product

Resources

About