No abstract

No abstract

A multi-armed bandit with finitely many arms is studied when each arm is a homogeneous Markov process on an underlying finite state space. The transition law of one of the arms, referred to as the odd arm, is different from the common transition law of all other arms. A learner, who has no knowledge of the above transition laws, has to devise a sequential test to identify the index of the odd arm as quickly as possible, subject to an upper bound on the probability of error. For this problem, we derive an asymptotic lower bound on the expected stopping time of any sequential test of the learner, where the asymptotics is as the probability of error vanishes. Furthermore, we propose a sequential test, and show that the asymptotic behaviour of its expected stopping time comes arbitrarily close to that of the lower bound. Prior works deal with independent and identically distributed arms, whereas our work deals with Markov arms. Our analysis of the rested Markov setting is a key first step in understanding the difficult case of restless Markov setting, which is still open. Index TermsMulti-armed bandits, rested bandits, Markov rewards, odd arm identification, anomaly detection, forced exploration.

The aim of this work is to establish that two recently published projection theorems, one dealing with a parametric generalization of relative entropy and another dealing with Rényi divergence, are equivalent under a correspondence on the space of probability measures. Further, we demonstrate that the associated "Pythagorean" theorems are equivalent under this correspondence. Finally, we apply Eguchi's method of obtaining Riemannian metrics from general divergence functions to show that the geometry arising from the above divergences are equivalent under the aforementioned correspondence.

In this work, we present a tool to construct and visualize the spatio-temporal variations of power. A dataset of real-world power measurements is collected over a geographical area of interest. Relevant parameters of the environment such as the path loss exponent and the decorrelation time of the lognormal shadow fading are extracted from the dataset. Also, the average powers measured at a finite set of known locations are interpolated to obtain the average power distribution over the area. Using the parameters of the lognormal shadow fading, synthetic data with the same temporal behavior of the dataset is generated, and multiplied with the average power distribution. The resulting spatio-temporal power map is displayed on the screen through a graphical user interface developed in-house. The proposed approaches for interpolation and parameter extraction are validated using test datasets generated using the well-accepted modified Gudmundson model for the spatio-temporal correlation of lognormal shadow fading. We also undertake a comparative study of three different interpolation techniques: linear interpolation, inverse distance weighing and ordinary kriging. Further, we compare a model-based approach with a model-free approach for interpolation, and find that model-based ordinary kriging provides the best mean absolute percentage error performance.

In this paper, we consider a multi-armed bandit in which each arm is a Markov process evolving on a finite state space. The state space is common across the arms, and the arms are independent of each other. The transition probability matrix of one of the arms (the odd arm) is different from the common transition probability matrix of all the other arms. A decision maker, who knows these transition probability matrices, wishes to identify the odd arm as quickly as possible, while keeping the probability of decision error small. To do so, the decision maker collects observations from the arms by pulling the arms in a sequential manner, one at each discrete time instant. However, the decision maker has a trembling hand, and the arm that is actually pulled at any given time differs, with a small probability, from the one he intended to pull. The observation at any given time is the arm that is actually pulled and its current state. The Markov processes of the unobserved arms continue to evolve. This makes the arms restless.For the above setting, we derive the first known asymptotic lower bound on the expected stopping time, where the asymptotics is of vanishing error probability. The continued evolution of each arm adds a new dimension to the problem, leading to a family of Markov decision problems (MDPs) on a countable state space. We then stitch together certain parameterised solutions to these MDPs and obtain a sequence of strategies whose expected stopping times come arbitrarily close to the lower bound in the regime of vanishing error probability. Prior works dealt with independent and identically distributed (across time) arms and rested Markov arms, whereas our work deals with restless Markov arms.

The pes are playing vital role in our day to day life. But also in the same time it emits lots of carbon equivalent in the atmosphere it results in global warming. The maj or emission of carbons is by E-waste and heat emitted by the servers that are in datacenters normally we all know computer emits lots of heat especially servers. For example, searching servers emit lots of heat per search. In this Research paper we provide a solution from the birth of the pc to degradation including solutions for the heat produced by the servers and this paper gives solution for power consumption. It also provides role of cloud computing in controlling carbon emission and saving of carbon. Due to the power consumption we saved the heat generated from the power stations.Here we providing the solution using fuzzy logic based devices with their working principles and more on of Power consumption Techniques. One more key thing from our proposal is it does not need any huge amount of investment and it is profitable for that concern also.

No abstract