A Deep Learning-Based Formula Detection Method for PDF Documents

Gao, Liangcai; Yi, Xiaosu; Liao, Yuan; Jiang, Zhuoren; Yan, Zuoyu; Tang, Zhi

doi:10.1109/icdar.2017.96

Cited by 33 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using ICDAR-2017 POD, their system failed to detect and highlight the missing or wrong labels, and it failed to capitalize on the ICDAR-2017 POD (corrected). It is worth emphasizing again that all the reported results on the original and updated annotations in this paper are generated using the official evaluation code released by organizers of the ICDAR-2017 competition on page object detection [18].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks

et al. 2020

View full text Add to dashboard Cite

We propose a novel hybrid approach that fuses traditional computer vision techniques with deep learning models to detect figures and formulas from document images. The proposed approach first fuses the different computer vision based image representations, i.e., color transform, connected component analysis, and distance transform, termed as Fi-Fo image representation. The Fi-Fo image representation is then fed to deep models for further refined representation-learning for detecting figures and formulas from document images. The proposed approach is evaluated on a publicly available ICDAR-2017 Page Object Detection (POD) dataset and its corrected version. It produces the state-of-the-art results for formula and figure detection in document images with an f1-score of 0.954 and 0.922, respectively. Ablation study results reveal that the Fi-Fo image representation helps in achieving superior performance in comparison to raw image representation. Results also establish that the hybrid approach helps deep models to learn more discriminating and refined features.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Gao et al [18] presented a combination of a convolutional neural network and recurrent neural network (RNN) models to detect formulas from PDF documents. A combination of CNN and RNN models enables this method to preserve both character and visual features for formula detection.…”

Section: Related Workmentioning

confidence: 99%

Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, He et al [36] exploited Convolutional Neural Networks (CNNs) with spatial context to detect mathematical symbols in document images. Later, Gao et al [37] presented a deep learning-based formula detection system in PDF documents. NLPR-PAL [27] produced the best results in the competition of POD at ICDAR-2017.…”

Section: Related Workmentioning

confidence: 99%

Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images

et al. 2021

View full text Add to dashboard Cite

This paper presents a novel architecture for detecting mathematical formulas in document images, which is an important step for reliable information extraction in several domains. Recently, Cascade Mask R-CNN networks have been introduced to solve object detection in computer vision. In this paper, we suggest a couple of modifications to the existing Cascade Mask R-CNN architecture: First, the proposed network uses deformable convolutions instead of conventional convolutions in the backbone network to spot areas of interest better. Second, it uses a dual backbone of ResNeXt-101, having composite connections at the parallel stages. Finally, our proposed network is end-to-end trainable. We evaluate the proposed approach on the ICDAR-2017 POD and Marmot datasets. The proposed approach demonstrates state-of-the-art performance on ICDAR-2017 POD at a higher IoU threshold with an f1-score of 0.917, reducing the relative error by 7.8%. Moreover, we accomplished correct detection accuracy of 81.3% on embedded formulas on the Marmot dataset, which results in a relative error reduction of 30%.

show abstract

“…The research [33] attempts to detect mathematical expressions in PDF documents by taking the advantages of CNNs. The framework for the detection consists of two steps.…”

Section: ) Mathematical Expression Detection In Native Pdf Documentsmentioning

confidence: 99%

A Hybrid Method for Mathematical Expression Detection in Scientific Document Images

Phong

Hoang

2020

IEEE Access

View full text Add to dashboard Cite

Mathematical expressions have been widely used in scientific documents. In order to analyze the documents, automatic detection of mathematical expressions is a crucial step. The paper presents a unified system for the detection of mathematical expressions including both inline and isolated expressions in scientific document images that usually consist of heterogeneous components (e.g., figures, tables, text and expressions). In the system, a hybrid method of two stages is proposed for the effective detection of mathematical expressions. First, the layout analysis of entire document images is introduced to improve the accuracy of text line and word segmentation. Then, both isolated and inline expressions in document images are detected. Both hand-crafted and deep learning features are extensively investigated and combined to improve the detection accuracy. Furthermore, a generic performance metric is applied to evaluate the system comprehensively. The proposed method has been evaluated on two public benchmark datasets (Marmot and GTDB). The obtained accuracies of isolated and inline expressions in the Marmot dataset are 91.18% and 81.35% while those in the GTDB dataset are 89.51% and 80.20%, respectively. The performance comparison is carried out with the conventional methods to show the outstanding effectiveness of the proposed system. Moreover, extensive experiments have been performed in order to point out the effect of document image resolution and post processing techniques on mathematical expression detection. INDEX TERMS Mathematical expression detection, document analysis, machine learning, neural network, fusion technique.

show abstract

A Deep Learning-Based Formula Detection Method for PDF Documents

Cited by 33 publications

References 9 publications

Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks

Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks

Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images

A Hybrid Method for Mathematical Expression Detection in Scientific Document Images

Contact Info

Product

Resources

About