Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models

Abstract

This paper describes a formal model for the recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Hidden Markov models are used to recognize mathematical symbols, and a stochastic context-free grammar is used to model the relation between these symbols. This formal model makes possible to use classic algorithms for parsing and stochastic estimation. In this way, first, the model is able to capture many of variability phenomena that appear in on-line handwritten mathematical expressions during the training process. And second, the parsing process can make decisions taking into account only stochastic information, and avoiding heuristic decisions. The proposed model participated in a contest of mathematical expression recognition and it obtained the best results at different levels.

Segmenting Handwritten Math Symbols Using AdaBoost and Multi-scale Shape Context Features

“…Another group of methods [11]- [15] are driven by grammars. There are some segmentation methods [11]- [13] under the restriction of a set of math expression grammars.…”

“…In [13], there is a set of fuzzy relational context free grammars. Some segmentation methods [14], [15] are driven by stochastic context free grammars (SCFG). In [14], the expressions recognition system is driven by the 2D contextfree probabilistic graph grammar to find all mathematically valid interpretations and assign scores to each possible interpretation.…”

Structure detection and segmentation of documents using 2D stochastic context-free grammars

“…SCFG are a powerful formalism of Syntactic Pattern Recognition which has been used previously for Document Image Analysis [20,21]. Bidimensional SCFG (2D-SCFG) is a well known formalism that has been studied in the past for bidimensional parsing [22,23]. This type of grammars are able to represent efficiently contextual bidimensional relations that are important for page segmentation [24].…”

Using Off-Line Features and Synthetic Data for On-Line Handwritten Math Symbol Recognition

“…In the survey by Yang et al [11] different feature extraction techniques for shape description have been identified. While some feature extraction techniques based on timing or drawing information have been used before for on-line symbol recognition [12], [13], [14], [15], [16], techniques based on shape description might be more robust since a single shape can be drawn in several ways. In the work by Delaye et al [17] a set of 49 features that describe both the drawing process and the shape have been proposed as a baseline for different on-line symbol recognition datasets.…”