Chart Mining: A Survey of Methods for Automated Chart Analysis

Davila, Kenny; Setlur, Srirangaraj; Doermann, David; Kota, Bhargava Urala; Govindaraju, Venu

doi:10.1109/tpami.2020.2992028

Cited by 64 publications

(42 citation statements)

References 125 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The captions and textual contents around figures are extracted by them as in [13]. Davila et al [14] observe extraction of chart data into two steps: document segmentation and linking figures to captions.…”

Section: B Chart Comprehensionmentioning

confidence: 99%

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Shahira

Lijiya

2021

IEEE Access

View full text Add to dashboard Cite

The textual data of a document is supplemented by the graphical information in it. To make communication easier, they contain tables, charts and images. However, it excludes a section of our population -the visually impaired. With technological advancements, the blind can access the documents through text to speech software solutions. In this method, even images can be conveyed by reading out the figure captions. However, charts and other statistical comparisons which involve critical information are difficult to be "read" out this way. Aim of this paper is to analyse various methods available to solve this vexatious issue. We survey the state-of-the-art works that do the exact opposite of graphing tools. In this paper, we explore the existing literature in understanding the graphs and extracting the visual encoding from them. We classify these approaches into modality-based approaches, conventional and deep-learning based methods. The survey also contains comparisons and analyses relevant study datasets. As an outcome of this survey, we observe that: (i) All existing works under each category need decoding in a variety of graphs. (ii) Among the approaches, deep learning performs remarkably well in localisation and classification. However, it needs further improvements in reasoning from chart images. (iii) Research works are still in progress to access data from vector images. Recreating data from the raster images has unresolved issues. Based on this study, the various applications of decoding the graphs, challenges and future possibilities are also discussed. This paper explores current works in the extraction of chart data, which seek to enable researchers in Human Computer Interaction to achieve human-level perception of visual data by machines. In this era of visual summarisation of data, the AI approaches can automate the underlying data extraction and hence provide the natural language descriptions to support visually disabled users.

show abstract

Section: B Chart Comprehensionmentioning

confidence: 99%

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Shahira

Lijiya

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Annotation was performed using crowdsourcing to reduce the time required. We employed annotators via the Amazon Mechanical Turk [4] to label the extracted graph images. To ensure the quality of the dataset, a maximum of nine annotators were used to annotate one figure.…”

Section: Annotation Detailsmentioning

confidence: 99%

“…First, we randomly sampled 100 figures in the dataset and annotated them using an annotator. We evaluated the annotations by comparing them with the labels of [3] https://www.lobe.ai/ [4] https://www.mturk.com/ the dataset. The results show that the accuracy of the labels in the dataset was 99% [5] , indicating that the labeling is accurate.…”

Section: Evaluation Of the Datasetmentioning

confidence: 99%

“…Several methods have recently been reported to automatically consume and codify information in figures of scientific literature across domains, such as image classification and optical character recognition [4,5], based on techniques adapted from the field of computer vision. These methods have immense potential to obtain data necessary for data-driven materials research based on the literature.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Annotation and Classification of Graphs of Property Values Reported in Material Science Literature

Iinuma

Kuniyoshi

Ozawa

et al. 2021

Preprint

View full text Add to dashboard Cite

Building a system for extracting information from the scientific literature is an important research topic in the field of inorganic materials science. However, conventional extracting systems have a limitation in that they do not extract characteristic values from non-textual components, such as charts, diagrams, and tables, which provide key information in many scientific documents. Although there have been several studies on identifying the characteristic values of graphs in the literature, there is no general method that classifies graphs according to the property conditions of the values in the field of materials science. Therefore, in this study, we focus on the graphs that are figures representing graphically numerical data, such as a bar graph and line graph, as the first step towards developing a framework for extracting material property information from such non-contextual components. We propose deep-learning-based classification models for identifying the types of graph properties, such as temperature and time, by combining graph images, text in graphs, and captions in neural networks. To train and evaluate the models, we construct a material graph dataset with different types of material properties from a large collection of data from journals in the field of materials science. By using cloud sourcing, we annotated 16,668 images in about 3 days. Our experimental results demonstrate that the best model can achieve high performance with a micro-averaged F-score of 0.961.

show abstract

“…Using these metrics of precision, however, both subfigures in Figure 4(a) and (b) will be considered "correct" in classification tasks, although they still demand subsequent algorithmic or human corrections. Our future work will study metrics for detailed evaluation, as processing compound figures remains one of the leading challenges in document analyses (Davila et al, 2020).…”

Section: Quantitative Evaluationmentioning

confidence: 99%

DeepPaperComposer: A Simple Solution for Training Data Preparation for Parsing Research Papers

Ling¹,

Chen²

2020

Proceedings of the First Workshop on Scholarly Document Processing

View full text Add to dashboard Cite

We present DeepPaperComposer, a simple solution for preparing highly accurate (100%) training data without manual labeling to extract content from scholarly articles using convolutional neural networks (CNNs). We used our approach to generate data and trained CNNs to extract eight categories of both textual (titles, abstracts, authors, headers, figure and table captions, and body texts) and nontextual content (figures and tables) from 30 years of 2916 IEEE VIS conference papers, of which a third were scanned bitmap PDFs. We curated this dataset and named it VISpaper-3K. We then showed our initial benchmark performance using VISpaper-3K over CS-150 using YOLOv3 and Faster-RCNN. We have opensourced DeepPaperComposer for training data generation 1 and have released the resulting annotation data VISpaper-3K 2 to promote reproducible research.

show abstract

Chart Mining: A Survey of Methods for Automated Chart Analysis

Cited by 64 publications

References 125 publications

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Annotation and Classification of Graphs of Property Values Reported in Material Science Literature

DeepPaperComposer: A Simple Solution for Training Data Preparation for Parsing Research Papers

Contact Info

Product

Resources

About