Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

Abstract

The textual data of a document is supplemented by the graphical information in it. To make communication easier, they contain tables, charts and images. However, it excludes a section of our population -the visually impaired. With technological advancements, the blind can access the documents through text to speech software solutions. In this method, even images can be conveyed by reading out the figure captions. However, charts and other statistical comparisons which involve critical information are difficult to be "read" out this way. Aim of this paper is to analyse various methods available to solve this vexatious issue. We survey the state-of-the-art works that do the exact opposite of graphing tools. In this paper, we explore the existing literature in understanding the graphs and extracting the visual encoding from them. We classify these approaches into modality-based approaches, conventional and deep-learning based methods. The survey also contains comparisons and analyses relevant study datasets. As an outcome of this survey, we observe that: (i) All existing works under each category need decoding in a variety of graphs. (ii) Among the approaches, deep learning performs remarkably well in localisation and classification. However, it needs further improvements in reasoning from chart images. (iii) Research works are still in progress to access data from vector images. Recreating data from the raster images has unresolved issues. Based on this study, the various applications of decoding the graphs, challenges and future possibilities are also discussed. This paper explores current works in the extraction of chart data, which seek to enable researchers in Human Computer Interaction to achieve human-level perception of visual data by machines. In this era of visual summarisation of data, the AI approaches can automate the underlying data extraction and hence provide the natural language descriptions to support visually disabled users.

Data Extraction and Question Answering on Chart Images Towards Accessibility and Data Interpretation

“…The paper [19] review the traditional methods and recent trends in the accessibility of visualization data and the potential future of artificial intelligence in reverse-engineering the chart data. The work, Revision [1] extracts low-level image features and classifies them using machine learning approach with 80% accuracy.…”

Image to audio, text to audio, text to speech, video to text conversion using, NLP techniques

“…In [1], This paper presents a method for defining the area of interest (ROI) in which objects can be segregated from messy backgrounds that is both efficient and effective. To obtain text information, this ROI extracts the text localization and recognition.…”

Data Extraction of Circular-Shaped and Grid-like Chart Images

“…For example, people with impaired vision and all blind individuals can not access the “locked” information inside chart images. These people often rely on assistive technology (braille display, screen readers, and speech converters), which can only read the information that is provided by the author in the surrounding text [ 3 , 4 ]. Manual reading of chart images is often inaccurate and can not be used for scientific purposes.…”