A comprehensive study of hybrid neural network hidden Markov model for offline handwritten Chinese text recognition

Wang, Zi-Rui; Du, Jun; Wang, Wenchao; Zhai, Jian-Fang; Hu, Jin-Shui

doi:10.1007/s10032-018-0307-0

Cited by 35 publications

(32 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, the authors in [16] proposed a novel aggregation cross-entropy loss for sequence recognition, which was shown to exhibit competitive performance for offline HCTR. In [17], we verified that combining hybrid deep CNN-HMM (DCNN-HMM) with a powerful language model could achieve the best reported results of the segmentation-free approaches on the ICDAR 2013 competition dataset.…”

Section: Introductionmentioning

confidence: 74%

“…In the CTC-based approach [15], a special character blank class and a defined many-to-one mapping function are introduced to directly compute p(C|X) with the forward-backward algorithm [13]. For the HMM-based approach [17], p(C|X) can be reformulated as the conditional probability p(X|C) and the prior probability p(C). More details will be provided in Section III.…”

Section: A Basic Principles For Offline Hctrmentioning

confidence: 99%

“…Aside from these methods, a key issue with CNN-HMM-based offline HCTR [17] is the large vocabulary problem, which leads to tens of thousands of output nodes (corresponding to HMM states) in CNN architecture. This heavy overhead in the output layer of the CNN not only requires high memory and computation costs but also yields more confusion among state classes and CNN training difficulties.…”

Section: B Model Compressionmentioning

confidence: 99%

“…Considering the particularity of HCTR and the difficulty of defining an effective question set for the Chinese language, in our previous work [31], we successfully invented a data-driven state-tying approach for a huge set of HMMs representing Chinese characters and achieved promising recognition performance. It should be noted that, if we simply reduce the state number for each character, the recognition accuracy will decline dramatically due to the lack of resolution for text line modeling [17].…”

Section: B Model Compressionmentioning

confidence: 99%

“…Our system follows the basic HMM framework [17] in which the handwritten text line is modeled by a series of cascading HMMs, each representing one character, as illustrated in Fig. 2.…”

Section: System Overviewmentioning

confidence: 99%

See 4 more Smart Citations

Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition

Wang

2020

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

Recently, the hybrid convolutional neural network hidden Markov model (CNN-HMM) has been introduced for offline handwritten Chinese text recognition (HCTR) and has achieved state-of-the-art performance. However, modeling each of the large vocabulary of Chinese characters with a uniform and fixed number of hidden states requires high memory and computational costs and makes the tens of thousands of HMM state classes confusing. Another key issue of CNN-HMM for HCTR is the diversified writing style, which leads to model strain and a significant performance decline for specific writers. To address these issues, we propose a writer-aware CNN based on parsimonious HMM (WCNN-PHMM). First, PHMM is designed using a data-driven state-tying algorithm to greatly reduce the total number of HMM states, which not only yields a compact CNN by state sharing of the same or similar radicals among different Chinese characters but also improves the recognition accuracy due to the more accurate modeling of tied states and the lower confusion among them. Second, WCNN integrates each convolutional layer with one adaptive layer fed by a writer-dependent vector, namely, the writer code, to extract the irrelevant variability in writer information to improve recognition performance.The parameters of writer-adaptive layers are jointly optimized with other network parameters in the training stage, while a multiple-pass decoding strategy is adopted to learn the writer code and generate recognition results. Validated on the ICDAR 2013 competition of CASIA-HWDB database, the more compact WCNN-PHMM of a 7360-class vocabulary can achieve a relative character error rate (CER) reduction of 16.6% over the conventional CNN-HMM without considering language modeling. By adopting a powerful hybrid language model (N-gram language model and recurrent neural network language model), the CER of WCNN-PHMM is reduced to 3.17%. Moreover, the state-tying results of PHMM explicitly show the information sharing among similar characters and the confusion reduction of tied state classes. Finally, we visualize the learned writer codes and demonstrate the strong relationship with the writing styles of different writers. To the best of our knowledge, WCNN-PHMM yields the 2 best results on the ICDAR 2013 competition set, demonstrating its power when enlarging the size of the character vocabulary. Index TermsOffline handwritten Chinese text recognition, writer-aware CNN, parsimonious HMM, state tying, adaptation, hybrid language model.

show abstract

Section: Introductionmentioning

confidence: 74%

Section: A Basic Principles For Offline Hctrmentioning

confidence: 99%

Section: B Model Compressionmentioning

confidence: 99%

Section: B Model Compressionmentioning

confidence: 99%

“…Our system follows the basic HMM framework [17] in which the handwritten text line is modeled by a series of cascading HMMs, each representing one character, as illustrated in Fig. 2.…”

Section: System Overviewmentioning

confidence: 99%

See 3 more Smart Citations