Multi-armed bandit based policies for cognitive radio's decision making issues

Jouini, Wassim; Ernst, Damien; Moy, Christophe; Palicot, Jacques

doi:10.1109/icscs.2009.5412697

Cited by 34 publications

(33 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However if the first phase is well achieved the second phase is usually very simple and does not require much time or energy [68]. In the second case (ii), we find promising techniques recently introduced to the community and still need to be further investigated [17,36] in the case of configuration adaptation. j These techniques try to provide the CR with a flexible and incremental learning decision making engine.…”

Section: Learning Approaches: Exploration and Exploitationmentioning

confidence: 99%

“…More recently, article [17] however assumes that no a priori knowledge is provided and that the performance of the equipment can only be estimated when trying a specific configuration. The associated tools are based on the so-called MAB framework.…”

Section: Learning Approaches: Exploration and Exploitationmentioning

confidence: 99%

“…We usually refer to this trade-off as the exploration-exploitation dilemma. If the problem is assumed modeled as a MAB framework an interesting way to tackle the problem is to use the class of so-called upper confidence bound algorithms n (UCB) [17,47,48,50,76]. The main advantage of UCB methods for CR is to offer a balance between exploration and exploitation phases without interrupting the communication process, i.e., while providing a certain service to the user [17].…”

Section: An Example Of Learning Approachmentioning

confidence: 99%

“…Since the original definition suggested by Joseph Mitola III, several other definitions were proposed to define the edges of CR [4,[8][9][10][15][16][17]. However, defining cognition is, in general, a harsh task.…”

Section: Basic Cognitive Cyclementioning

confidence: 99%

See 3 more Smart Citations

Decision making for cognitive radio equipment: analysis of the first 10 years of exploration

Jouini

Moy

Palicot

2012

J Wireless Com Network

Self Cite

View full text Add to dashboard Cite

This article draws a general retrospective view on the first 10 years of cognitive radio (CR). More specifically, we explore in this article decision making and learning for CR from an equipment perspective. Thus, this article depicts the main decision making problems addressed by the community as general dynamic configuration adaptation (DCA) problems and discuss the suggested solution proposed in the literature to tackle them. Within this framework dynamic spectrum management is briefly introduced as a specific instantiation of DCA problems. We identified, in our analysis study, three dimensions of constrains: the environment's, the equipment's and the user's related constrains. Moreover, we define and use the notion of a priori knowledge, to show that the tackled challenges by the radio community during first 10 years of CR to solve decision making problems have often the same design space, however they differ by the a priori knowledge they assume available. Consequently, we suggest in this article, the "a priori knowledge" as a classification criteria to discriminate the main proposed techniques in the literature to solve configuration adaptation decision making problems. We finally discuss the impact of sensing errors on the decision making process as a prospective analysis.

show abstract

Section: Learning Approaches: Exploration and Exploitationmentioning

confidence: 99%

Section: Learning Approaches: Exploration and Exploitationmentioning

confidence: 99%

Section: An Example Of Learning Approachmentioning

confidence: 99%

Section: Basic Cognitive Cyclementioning

confidence: 99%

See 2 more Smart Citations

Decision making for cognitive radio equipment: analysis of the first 10 years of exploration

Jouini

Moy

Palicot

2012

J Wireless Com Network

Self Cite

View full text Add to dashboard Cite

show abstract

“…At the equipment level, we foresee many opportunities from neuronal networks to Markov decision processes. A CR equipment has to estimate the environment even if only a partial knowledge is obtained [37]. This will be often the case in a CR context as we expect to take into account many different parameters.…”

Section: Examples Of Cognitive Decisionsmentioning

confidence: 99%

High-Level Design Approach for the Specification of Cognitive Radio Equipments Management APIs

Moy

2009

J Netw Syst Manage

Self Cite

View full text Add to dashboard Cite

Cognitive Radio (CR) equipments are radio devices that support the smart facilities offered by future cognitive networks. Even if several categories of equipments exist (terminal, base station, smart PDA, etc.), each requiring different processing capabilities (and associated cost or power consumption), these equipments have to integrate also a set of new capabilities as regards CR support, in addition to the usual radio signal processing elements. This implies real-time radio adaptation and sensing capabilities, but not only. We assert that it is necessary to add inside the radio equipments some management facilities for that purpose, and we propose in this paper a high-level design approach for the specification of a management framework. This includes a set of designing rules, based on hierarchical units that are distributed over three levels, and the associated APIs necessary to efficiently manage CR features inside a CR equipment. The proposed architecture is called HDCRAM (Hierarchical and Distributed Cognitive Architecture Management). HDCRAM is an extension of a former hierarchical and distributed reconfiguration management (HDReM) architecture, which is derived from our previous research on Software Defined Radio (SDR). The HDCRAM adds to the HDReM's reconfiguration management facilities the necessary new management features, which enable the support of sensing and decision making facilities. It consists in the combination of one Cognitive Radio Management Unit (CRMU) with eachReconfiguration Management Unit (ReMU) IntroductionThe design of cognitive radio (CR) equipments will not be done efficiently if one uses the same tools as those used for the design of conventional radio equipments. One reason is that designing such a radio device is not solely a radio signal processing issue anymore. Of course, as in many other industrial domains, such as aeronautics, it becomes a hardware/software mixed issue with a plurality of capabilities and running modes on the one hand. But on the other hand, designing cognitive radio equipments also consists in designing the necessary management needed to support the real-time sensing of the environment and the real-time adaptation of the signal processing. Signal processing in our approach is not restricted to the physical layer and also includes all layers' signal processing, up to the application layer. Another reason is that the range of ways a CR equipment will have to run, will be very wide compared to that of the current radio equipment operation. Abstraction is necessary during the design phase of a CR equipment. In this respect, this paper proposes a high-level design approach, 2 integrating the design rules needed for the management of the signal processing operators, as well as the exploration means as regards the operation modes.Even if designing the signal processing part of a radio chain is not an easy task, it is however usual. The new design aspect lies in the management part and the associated impact on the signal processing, which has to be fl...

show abstract

Concurrent Bandits and Cognitive Radio Networks

Avner

Mannor

2014

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

We consider the problem of multiple users targeting the arms of a single multi-armed stochastic bandit. The motivation for this problem comes from cognitive radio networks, where selfish users need to coexist without any side communication between them, implicit cooperation or common control. Even the number of users may be unknown and can vary as users join or leave the network. We propose an algorithm that combines an -greedy learning rule with a collision avoidance mechanism. We analyze its regret with respect to the systemwide optimum and show that sub-linear regret can be obtained in this setting. Experiments show dramatic improvement compared to other algorithms for this setting.

show abstract

Multi-armed bandit based policies for cognitive radio's decision making issues

Cited by 34 publications

References 13 publications

Decision making for cognitive radio equipment: analysis of the first 10 years of exploration

Decision making for cognitive radio equipment: analysis of the first 10 years of exploration

High-Level Design Approach for the Specification of Cognitive Radio Equipments Management APIs

Concurrent Bandits and Cognitive Radio Networks

Contact Info

Product

Resources

About