The recent emergence of reinforcement learning has created a demand for robust statistical inference methods for the parameter estimates computed using these algorithms. Existing methods for statistical inference in online learning are restricted to settings involving independently sampled observations, while existing statistical inference methods in reinforcement learning (RL) are limited to the batch setting. The online bootstrap is a flexible and efficient approach for statistical inference in linear stochastic approximation algorithms, but its efficacy in settings involving Markov noise, such as RL, has yet to be explored. In this paper, we study the use of the online bootstrap method for statistical inference in RL. In particular, we focus on the temporal difference (TD) learning and Gradient TD (GTD) learning algorithms, which are themselves special instances of linear stochastic approximation under Markov noise. The method is shown to be distributionally consistent for statistical inference in policy evaluation, and numerical experiments are included to demonstrate the effectiveness of this algorithm at statistical inference tasks across a range of real RL environments.
This paper presents a new technique to assess the bone formation after periapical dental surgery. The proposed technique consists of two main stages: image registration and spectral subtraction stages. Image registration is used to avoid projection errors produced due to nonstandardization of X-ray scanners. Wavelet coefficients are used instead of grey values for registering the images. Coarse to fine strategy with four levels of resolutions is used to speed up the process. The second stage is the spectral subtraction stage. It is used to yield the difference image between pre-and post-operative images which represents the bone gain or bone loss with light and dark areas, respectively. Algorithm has been applied on a number of pre-and post-surgery intra oral periapical (IOP) dental X-ray images. Mean and root mean square error (RMSE) are computed to assess the quality of registration technique. The technique presented here is compared with grey level based method; results show that proposed technique outperforms conventional grey level method based on dyadic sampling.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.