On the complete convergence for martingale difference sequence

Chen, Ying-Xia; Zhang, Shuili; Ma, Fuqiang

doi:10.1080/03610926.2016.1157188

Cited by 4 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since ( 4) and ( 5) hold true, the sequence of empirical errors {[( Pki h − P h )V * h+1 ](x, a)} K i=1 can be interpreted as a martingale difference sequence (MDS) with respect to the filtration {F} K i=1 [37]. Therefore, we can use the Azuma-Hoeffding inequality to give a concentration result [38] for each index in the MDS, i.e., to construct confidence bounds for Q * h ∀ h ∈ {1, 2, .…”

Section: From the Error [( Pkimentioning

confidence: 99%

Is Q-Learning Provably Efficient? An Extended Analysis

Rastogi,

Lee,

Harel-Canada

et al. 2020

Preprint

View full text Add to dashboard Cite

This work extends the analysis of the theoretical results presented within the paper Is Q-Learning Provably Efficient? by Jin et al. [1]. We include a survey of related research to contextualize the need for strengthening the theoretical guarantees related to perhaps the most important threads of model-free reinforcement learning. We also expound upon the reasoning used in the proofs to highlight the critical steps leading to the main result showing that Q-learning with UCB exploration achieves a sample efficiency that matches the optimal regret that can be achieved by any model-based approach.

show abstract

Section: From the Error [( Pkimentioning

confidence: 99%

Is Q-Learning Provably Efficient? An Extended Analysis

Rastogi,

Lee,

Harel-Canada

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Miao et al [ 5 ] improved some known results and studied the Baum–Katz type convergence rate in the Marcinkiewicz–Zygmund strong law for martingales. Chen et al [ 6 ] also gave some extended results for the sequence of martingale difference.…”

Section: Introductionmentioning

confidence: 99%