In online learning problems, exploiting low variance plays an important role in obtaining tight performance guarantees yet is challenging because variances are often not known a priori. Recently, a considerable progress has been made by Zhang et al. (2021) where they obtain a variance-adaptive regret bound for linear bandits without knowledge of the variances and a horizon-free regret bound for linear mixture Markov decision processes (MDPs). In this paper, we present novel analyses that improve their regret bounds significantly. For linear bandits, we achieve Õ(d 1.5 ∑ K k σ 2 k +d 2 ) where d is the dimension of the features, K is the time horizon, and σ 2 k is the noise variance at time step k, and Õ ignores polylogarithmic dependence, which is a factor of d 3 improvement. For linear mixture MDPs, we achieve a horizon-free regret bound of Õ(d 1.5 √ K +d 3 ) where d is the number of base models and K is the number of episodes. This is a factor of d 3 improvement in the leading term and d 6 in the lower order term. Our analysis critically relies on a novel elliptical potential 'count' lemma. This lemma allows a peeling-based regret analysis, which can be of independent interest.
The goal of this paper is to study a Hamilton-Jacobi equationwith initial conditions I(0) = 0, u 0 (x, 0) = u 0 (x) on R n . Here (u, I) is a pair of unknowns and the Hamiltonian H and the reaction R are given. And I(t) is an unknown constraint (Lagrange multiplier) that forces supremum of u to be always zero. We construct a solution in the viscosity setting using a fixed point argument when the reaction term R(x, I) is strictly decreasing in I. We also discuss both uniqueness and nonuniqueness. For uniqueness, a certain structural assumption on R(x, I) is needed. We also provide an example with infinitely many solutions when the reaction term is not strictly decreasing in I.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.