A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN models

Truong, Thanh-Nghia; Nguyen, Cuong Tuan; Zanibbi, Richard; Mouchère, Harold; Nakagawa, Masaki

doi:10.1016/j.patcog.2024.110531

Pattern Recognition

2024

DOI: 10.1016/j.patcog.2024.110531

|View full text |Cite

A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN models

Thanh-Nghia Truong,

Cuong Tuan Nguyen,

Richard Zanibbi

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 99 publications

(221 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

GLAN: A graph-based linear assignment network

Liu,

Wang,

Lang

et al. 2024

Pattern Recognition

View full text Add to dashboard Cite

GLAN: A graph-based linear assignment network

Liu,

Wang,

Lang

et al. 2024

Pattern Recognition

View full text Add to dashboard Cite

Improving Handwritten Mathematical Expression Recognition via Integrating Convolutional Neural Network With Transformer and Diffusion-Based Data Augmentation

Zhang,

2024

IEEE Access

View full text Add to dashboard Cite

Handwritten mathematical expression recognition (HMER) poses a formidable challenge due to the intricate two-dimensional structures and diverse handwriting styles. This paper introduces a novel approach to improve HMER accuracy by employing an integrated, high-capacity architecture that combines Transformer and Convolutional Neural Network (CNN) models, along with a denoising diffusion probabilistic model (DDPM)-based data augmentation technique. We explore three combination strategies for an attention-based encoder-decoder (AED) HMER model: 1) The ''Tandem'' strategy, which harnesses CNN features within a Transformer encoder to capture global interdependencies; 2) The ''Parallel'' strategy, which integrates Transformer encoder outputs with CNN outputs to achieve comprehensive feature fusion; 3) The ''Mixing'' strategy, which introduces multi-head self-attention (MHSA) at the final stage of the CNN. We evaluate these methods using the CROHME benchmark dataset and conduct a detailed comparative analysis. All three approaches significantly enhance model performance. Notably, the ''Tandem'' approach achieves expression recognition rates (ExpRate) of 54.85% and 58.56% on the CROHME 2016 and 2019 test sets, respectively, while the ''Parallel'' method attains 55.63% and 57.39% on the same test sets. Furthermore, we introduce an innovative data augmentation approach that utilizes DDPM to generate synthetic training samples. The DDPM, conditioned on LaTeX-rendered images, bridges the gap between printed and handwritten expressions, enabling the creation of realistic, stylistically diverse handwriting samples. This augmentation boosts the ExpRates of all strategies on both CROHME 2016 and 2019 test sets, yielding improvements of 1.6-4.6% relative to the unaugmented dataset.

show abstract

Research on safety accident risk of construction engineering based on natural language processing technology and neural network technology

Wang,

Meng

2024

Applied Mathematics and Nonlinear Sciences

View full text Add to dashboard Cite

In recent years, China’s construction industry has achieved rapid development, while the problem of safety accident risks brought by construction projects has also arisen. In this paper, based on natural language processing technology, using a graph neural network algorithm, we constructed a graph data model of safety accident reports in construction engineering and generated a database of association rules of safety risk factors in construction engineering to identify and classify the risk of safety accidents in actual construction engineering. The pictures of safety risk behaviors at the construction site are processed with feature mapping and inputted into a convolutional neural network to build a construction engineering safety warning model. Through simulation experiments, the identification results of risk factors for the model constructed in this paper and the effective degree of safety warning are tested. The top five factors in the centrality ranking of risk factors of construction engineering safety accidents are S17, S6, S1, S9, and S8, with centrality levels of 1.9779, 1.8626, 1.8204, 1.6332 and 1.4534, and the factor attribute of S6 is the outcome factor. The warning model constructed in this paper is measured against the level of safety risk of the construction project under the non-adoption of the warning model. The safety risk is reduced by 35.8% after the adoption of the safety accident warning model, which shows that the model in this paper has a certain inhibition effect on the risk of safety accidents.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN models

Cited by 3 publications

References 99 publications

GLAN: A graph-based linear assignment network

GLAN: A graph-based linear assignment network

Improving Handwritten Mathematical Expression Recognition via Integrating Convolutional Neural Network With Transformer and Diffusion-Based Data Augmentation

Research on safety accident risk of construction engineering based on natural language processing technology and neural network technology

Contact Info

Product

Resources

About