Document Image Classification: Towards Assisting Visually Impaired

Shahira, K. C.; Lijiya, A

doi:10.1109/tencon.2019.8929594

Cited by 8 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extracting data from the chart images has been extensively studied recently. Some works concentrate on recognizing the figure in a document as a graphical image [45] or not, while others classify it into various types of charts [4], [28], [29], [39], whereas others concentrate on extracting the visual elements from the charts [26], [40], [46]. We categorise the methods in extracting the underlying data as modality based methods, traditional and deep learning-based methods.…”

Section: Methods For Identifying Chartsmentioning

confidence: 99%

“…They can be part of textbooks, newspapers, scientific studies and other statistical studies. The problem of understanding charts by the visually impaired and the state-of-the-art methods in identifying the chart types is studied in [4]. The charts or any information is accessible to the visually disabled in the form of touch, audio or both.…”

Section: B Chart Comprehensionmentioning

confidence: 99%

“…Any of them can be fine-tuned towards a specific type of classification task. Classifying the figures in documents into different chart types using CNN, which is similar in structure to an AlexNet [70] is used in [4]. Motivated by the positive results of a classification in natural images by CNN, Table 4 summarises the various network architectures designed and the pre-trained models used for recognising the charts images.…”

Section: ) Localisation and Chart Type Identificationmentioning

confidence: 99%

See 2 more Smart Citations

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Shahira

Lijiya

2021

IEEE Access

View full text Add to dashboard Cite

The textual data of a document is supplemented by the graphical information in it. To make communication easier, they contain tables, charts and images. However, it excludes a section of our population -the visually impaired. With technological advancements, the blind can access the documents through text to speech software solutions. In this method, even images can be conveyed by reading out the figure captions. However, charts and other statistical comparisons which involve critical information are difficult to be "read" out this way. Aim of this paper is to analyse various methods available to solve this vexatious issue. We survey the state-of-the-art works that do the exact opposite of graphing tools. In this paper, we explore the existing literature in understanding the graphs and extracting the visual encoding from them. We classify these approaches into modality-based approaches, conventional and deep-learning based methods. The survey also contains comparisons and analyses relevant study datasets. As an outcome of this survey, we observe that: (i) All existing works under each category need decoding in a variety of graphs. (ii) Among the approaches, deep learning performs remarkably well in localisation and classification. However, it needs further improvements in reasoning from chart images. (iii) Research works are still in progress to access data from vector images. Recreating data from the raster images has unresolved issues. Based on this study, the various applications of decoding the graphs, challenges and future possibilities are also discussed. This paper explores current works in the extraction of chart data, which seek to enable researchers in Human Computer Interaction to achieve human-level perception of visual data by machines. In this era of visual summarisation of data, the AI approaches can automate the underlying data extraction and hence provide the natural language descriptions to support visually disabled users.

show abstract

Section: Methods For Identifying Chartsmentioning

confidence: 99%

Section: B Chart Comprehensionmentioning

confidence: 99%

Section: ) Localisation and Chart Type Identificationmentioning

confidence: 99%

See 1 more Smart Citation

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Shahira

Lijiya

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This paper aims to extract the data from the bar chart images and make the chart data available, thus making it accessible to all and also by search engines. Towards achieving this goal, the following contributions have been made: Once the images are classified into their respective categories [13], [14], the data extraction begins. In this section, we review the related works in chart comprehension in data extraction and question answering on charts as follows:…”

Section: A Contributions Of This Papermentioning

confidence: 99%

Data Extraction and Question Answering on Chart Images Towards Accessibility and Data Interpretation

Joshi,

2023

IEEE Open J. Comput. Soc.

View full text Add to dashboard Cite

Graphical representations such as chart images are integral to web pages and documents. Automating data extraction from charts is possible by reverse-engineering the visualization pipeline. This study proposes a framework that automates data extraction from bar charts and integrates it with questionanswering. The framework employs an object detector to recognize visual cues in the image, followed by text recognition. Mask-RCNN for plot element detection achieves a mean average precision of 95.04% at a threshold of 0.5 which decreases as the Intersection over Union (IoU) threshold increases. A contour approximation-based approach is proposed for extracting the bar coordinates, even at a higher IoU of 0.9. The textual and visual cues are associated with the legend text and preview, and the chart data is finally extracted in tabular format. We introduce an extension to the TAPAS model, called TAPAS++, by incorporating new operations and table question answering is done using TAPAS++ model. The chart summary or description is also produced in an audio format. In the future, this approach could be expanded to enable interactive question answering on charts by accepting audio inquiries from individuals with visual impairments and do more complex reasoning using Large Language Models.

show abstract

“…Customized charts tailored to specific data types enhance structural clarity. The initial step in interpreting visualizations involves chart classification, followed by processing techniques like data export algorithms and optical character recognition systems [3,4,5].…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Shallow Convolutional Neural Network in Open-World Chart Image Classification

Bajić,

Habijan,

Nenadić

2024

IJCAI

View full text Add to dashboard Cite

Data's role is pivotal in the era of internet technologies, but unstructured data poses comprehension challenges. Data visualizations like charts have emerged as crucial tools for condensing complex information. Classifying charts and applying various processing techniques are vital to interpreting visual data. Traditional chart image classification methods rely on predefined rules and have limited accuracy. The advent of support vector machines (SVMs) and convolutional neural networks (CNNs) significantly improved the accuracy of these methods. This research evaluates our previously introduced Shallow convolutional neural network (SCNN) architecture for chart image classification, comprising four convolutional layers, two max-pooling layers, and one fully-connected layer. The network achieves state-of-the-art results, requiring smaller datasets and reduced computational resources. When two networks are combined into Siamese SCNN (SSCNN), emphasizing generalization, it achieves high accuracy with small datasets and excels in open-set classification. The evaluation process encompasses the utilization of six publicly available datasets.Povzetek: Raziskava uvaja arhitekturo plitkega konvolucijskega nevronskega omrežja (SCNN) za klasifikacijo slik grafikonov.

show abstract

Document Image Classification: Towards Assisting Visually Impaired

Cited by 8 publications

References 10 publications

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Data Extraction and Question Answering on Chart Images Towards Accessibility and Data Interpretation

Evaluation of Shallow Convolutional Neural Network in Open-World Chart Image Classification

Contact Info

Product

Resources

About