Semi-supervised Online Learning of Handwritten Characters Using a Bayesian Classifier

Abstract

This paper addresses the problem of creating a handwritten character recognisor, which makes use of both labelled and unlabelled data to learn continuously over time to make the recognisor adaptable. The proposed method makes learning possible from a continuous inflow of a potentially unlimited amount of data without the requirement for storage. It highlights the use of unlabelled data for better parameter estimation, especially when labelled data is scarce and expensive unlike unlabelled data. We introduce an algorithm for learning from labelled and unlabelled samples based on the combination of novel online ensemble of the Randomized Naive Bayes classifiers and a novel incremental variant of the Expectation Maximization (EM) algorithm. We make use of a weighting factor to modulate the contribution of unlabelled data. An empirical evaluation of the proposed method on Tamil handwritten base character recognition proves efficacy of the proposed method to carry out incremental semi-supervised learning and producing accuracy comparable to state-of-the-art batch learning method. Online handwritten Tamil characters from the IWFHR 2006 competition dataset was used for evaluating the proposed method.

Semi-supervised Online Bayesian Network Learner for Handwritten Characters Recognition

“…Analysis has been done over the static λ in [7] and results have shown that, how its large value can otherwise hurt the accuracy of the classifier. On the similar line we have studied the dynamic λ k .…”

“…The most close work present in the literature is by Kunwar et al [7] but there exist some striking differences for eg. the classifier used is different, the proposed online variant of Expectation Maximization (EM) used is also different.…”

“…Though it is inspired by the growing GMM proposed in [2] or online EM proposed in [7], there exist the following differences: a) the learning rate estimation step, in the present work we use a varying weighting factor (learning rate) which helps moderate the contribution of unlabelled data with time in the M step and hence prevents model parameters from generating improper estimates, b) estimation of variance in the M step, c) the capability to handle both labelled and unlabelled data together.…”

“…Among one of the previous attempts to use randomization with naive Bayes (the most basic version of BN) was Prinzie et al [12]. Where as fair amount of attempts have been made to make the online learning of learning parameters [14] [7]. We propose a method to update the values of learning parameters of BN in online fashion.…”

“…Initialization of the learning parameters is done by 1 sample as given in [7]. Repeat, steps below for all the available labelled training samples:…”