Edge-reinforced random walk (ERRW), introduced by Coppersmith and Diaconis in 1986 [8], is a random process, which takes values in the vertex set of a graph G, and is more likely to cross edges it has visited before. We show that it can be represented in terms of a Vertex-reinforced jump process (VRJP) with independent gamma conductances: the VRJP was conceived by Werner and first studied by Davis and Volkov [10,11], and is a continuous-time process favouring sites with more local time. We calculate, for any finite graph G, the limiting measure of the centred occupation time measure of VRJP, and interpret it as a supersymmetric hyperbolic sigma model in quantum field theory [16]. This enables us to deduce that VRJP and ERRW are positive recurrent in any dimension for large reinforcement, and that VRJP is transient in dimension greater than or equal to 3 for small reinforcement, using results of Disertori and Spencer [15], Disertori, Spencer and Zirnbauer [16].

In this paper, an online learning algorithm is proposed as sequential stochastic approximation of a regularization path converging to the regression function in reproducing kernel Hilbert spaces (RKHSs). We show that it is possible to produce the best known strong (RKHS norm) convergence rate of batch learning, through a careful choice of the gain or step size sequences, depending on regularity assumptions on the regression function. The corresponding weak (mean square distance) convergence rate is optimal in the sense that it reaches the minimax and individual lower rates in the literature. In both cases we deduce almost sure convergence, using Bernstein-type inequalities for martingales in Hilbert spaces.To achieve this we develop a bias-variance decomposition similar to the batch learning setting; the bias consists in the approximation and drift errors along the regularization path, which display the same rates of convergence, and the variance arises from the sample error analysed as a reverse martingale difference sequence. The rates above are obtained by an optimal trade-off between the bias and the variance.

International audienc

We investigate the asymptotic behavior of one version of the so-called two-armed bandit algorithm. It is an example of stochastic approximation procedure whose associated ODE has both a repulsive and an attractive equilibrium, at which the procedure is noiseless. We show that if the gain parameter is constant or goes to 0 not too fast, the algorithm does fall in the noiseless repulsive equilibrium with positive probability, whereas it always converges to its natural attractive target when the gain parameter goes to zero at some appropriate rates depending on the parameters of the model. We also elucidate the behavior of the constant step algorithm when the step goes to 0. Finally, we highlight the connection between the algorithm and the Polya urn. An application to asset allocation is briefly described

Vertex-reinforced random walk (VRRW), defined by Pemantle in 1988, is a random process that takes values in the vertex set of a graph G, which is more likely to visit vertices it has visited before. Pemantle and Volkov considered the case when the underlying graph is the one-dimensional integer lattice Z. They proved that the range is almost surely finite and that with positive probability the range contains exactly five points. They conjectured that this second event holds with probability 1. The proof of this conjecture is the main purpose of this paper.In other words, moves are restricted to the edges of G, with the probability of a move to a neighbor x being proportional to the augmented occupation Z n (x) of x at that time.VRRWs were introduced in 1988 by Pemantle [7] in the spirit of the seminal work by Coppersmith and Diaconis [4], who defined the notion of edge-reinforced random walks, which have at each step a probability to move along an edge proportional to the number of times plus 1 that the process has visited this edge. Reinforced processes are useful in models involving self-organization and learning behavior; they can also describe spatial monopolistic competition in economics. For more details on applications and known results in connection with these models, refer to the articles by Pemantle and Volkov [8,9].VRRWs on finite complete graphs, with reinforcements weighted by factors associated to each edge of the graph, have been studied by Pemantle [8] and Benam [1]. Pemantle and Volkov obtained results in 1997 on reinforced random walks on Z [9], which are described in the following text. More recently, Volkov [13] generalized some of these results and proved that, on a fairly broad class of locally finite graphs (containing the graphs of bounded degree), the VRRW has finite range with positive probability. The remainder of this paper is devoted to VRRWs on Z.Define the two random sets R := {v ∈ Z/ ∃ n ∈ N s.t. X n = v}, R ′ := {v ∈ Z/X n = v infinitely often} and, given k ∈ Z and α ∈ (0, 1), define the six events:Let | · | be the cardinality of a set. Pemantle and Volkov [9] proved the following results.Theorem 1.1. One has P(|R| < ∞) = 1 and P(|R| = 5) > 0.Theorem 1.2. One has P(|R ′ | ≤ 4) = 0. Theorem 1.3. For any open set I ⊂ [0, 1] and any integer k ∈ Z there exists, with positive probability, α ∈ I such that events 1-6 occur.Pemantle and Volkov also proposed the following conjecture.