Handwritten Chinese/Japanese Text Recognition Using Semi-Markov Conditional Random Fields

Abstract

This paper proposes a method for handwritten Chinese/Japanese text (character string) recognition based on semi-Markov conditional random fields (semi-CRFs). The high-order semi-CRF model is defined on a lattice containing all possible segmentation-recognition hypotheses of a string to elegantly fuse the scores of candidate character recognition and the compatibilities of geometric and linguistic contexts by representing them in the feature functions. Based on given models of character recognition and compatibilities, the fusion parameters are optimized by minimizing the negative log-likelihood loss with a margin term on a training string sample set. A forward-backward lattice pruning algorithm is proposed to reduce the computation in training when trigram language models are used, and beam search techniques are investigated to accelerate the decoding speed. We evaluate the performance of the proposed method on unconstrained online handwritten text lines of three databases. On the test sets of databases CASIA-OLHWDB (Chinese) and TUAT Kondate (Japanese), the character level correct rates are 95.20 and 95.44 percent, and the accurate rates are 94.54 and 94.55 percent, respectively. On the test set (online handwritten texts) of ICDAR 2011 Chinese handwriting recognition competition, the proposed method outperforms the best system in competition.

A unified method for augmented incremental recognition of online handwritten Japanese and English text

“…We used a trigram table extracted from the year 1993 volume of the Asahi newspaper and the year 2002 volume of the Nikkei newspaper to model linguistic context. From the TUAT-Kondate database collected from 100 people [12], we separated the text lines into 4 sets by writers and then used 3 sets (10,174 text lines written by 75 people) for training the weighting parameters and 1 set (3511 text lines written by 25 people) for testing as in [27,31]. We changed the role four times and took the average.…”

“…Therefore, recognition time grows exponentially as the length of input sequence increases. To reduce recognition time for handwritten Chinese and Japanese text, candidate character patterns formed by multiple primitive segments have been restricted in length [27,31]. The length restriction, however, is not applicable for handwritten English text due to a large variance in the lengths of candidate word patterns.…”

“…Owing to the progress in deep neural network technology, one can consider deploying it for practical systems, but there are some obstacles such as speed and memory space for the large category size to be used in stand-alone systems, especially for hand-held mobile phones and tablets. On the other hand, the explicit segmentation technique also provides reliable performance in recognition of online handwritten Japanese text [31], and online handwritten Chinese text [27]. This approach is also applied for online handwritten English text recognition [19,23].…”

Generative vs. Discriminative Recognition Models for Off-Line Arabic Handwriting

“…The recommended method was tested on dual handwriting datasets and reportedly outperformed methods that adopt shallow CRF. Zhou et al [ 17 ] proposed a method for detecting Japanese and Chinese text that accorded with semi-Markov CRF. The researchers began with descriptions of semi-CRF on lattices comprised of every possible segmentation-recognition hypothesis of strings, in order to directly approximate the a posteriori probabilities for each.…”

“…Due to the fact that HMM, CRF and HCRF are especially powerful for the task of sequential feature classification, features extracted from a letter or a word image should be sequential or can be easily converted into a sequence that eventually is passed to the appropriate classifier for recognition. The most widely-adopted sequential features for offline handwriting recognition are those extracted using the principle of the so-called sliding window [ 6 , 10 , 14 , 17 ]. Typically, these types of features are sequences of observations extracted by shifting a window along the image of the word from right to left or vice versa.…”