A Model-Based Sequence Similarity with Application to Handwritten Word Spotting

Abstract

This paper proposes a novel similarity measure between vector sequences. We work in the framework of model-based approaches, where each sequence is first mapped to a Hidden Markov Model (HMM) and then a measure of similarity is computed between the HMMs. We propose to model sequences with semicontinuous HMMs (SC-HMMs). This is a particular type of HMM whose emission probabilities in each state are mixtures of shared Gaussians. This crucial constraint provides two major benefits. First, the a priori information contained in the common set of Gaussians leads to a more accurate estimate of the HMM parameters. Second, the computation of a similarity between two SC-HMMs can be simplified to a Dynamic Time Warping (DTW) between their mixture weight vectors, which significantly reduces the computational cost. Experiments are carried out on a handwritten word retrieval task in three different datasets-an in-house dataset of real handwritten letters, the George Washington dataset, and the IFN/ENIT dataset of Arabic handwritten words. These experiments show that the proposed similarity outperforms the traditional DTW between the original sequences, and the model-based approach which uses ordinary continuous HMMs. We also show that this increase in accuracy can be traded against a significant reduction of the computational cost.

Handwritten Word Spotting with Corrected Attributes

“…We first compare with our reimplementation of the exemplar SVM-based approach of [1], where, at query time, a classifier is trained using the query as a positive sample and the training set as negative samples, and the model is used to rank the dataset. We also compare [25] are not exactly comparable, we use partitions of the same size and very similar protocols. We also report results using the character HMM of [7], as well as the results reported on [9] using that method with a simpler subset.…”

“…Note that this approach restricts one to keywords that need to be learned offline, usually with large amounts of data. In [25], this problem is solved by learning a semicontinuous HMM (SC-HMM). The parameters of the SC-HMM are learned on a pool of unsupervised samples.…”

“…Variable-length features are more flexible than feature vectors and have been known to lead to superior results in difficult word-spotting tasks since they can adapt better to the different variations of style and word length [7,9,22,24,25]. Unfortunately, this leads to two unsatisfying outcomes.…”

“…Second, because the methods deal with sequences of features, computing distances between words is usually very slow at test time. As a consequence, approaches such as exhaustive sliding window search are not feasible, and the words in the dataset documents need to be segmented [7,22,24,25]. In many real scenarios this is error prone and time consuming, and can be just unfeasible if the dataset is large enough.…”

Efficient Exemplar Word Spotting

“…Finally, by using a similarity measure -commonly a Dynamic Time Warping (DTW) or a Hidden Markov Model (HMM)-based similarity -, the query word is compared and candidates are ranked according to this similarity. Examples of this framework are the works of Rath and Manmatha [16] and Rodríguez-Serrano and Perronnin [18]. One of the main drawbacks of these systems is that they need to perform a costly and error prone segmentation step to select candidate windows.…”