OpenCQA: Open-ended Question Answering with Charts

NLP+Vis: NLP Meets Visualization

“…While visualizations can be very effective in finding patterns, trends, and outliers in data, natural language can help explain the key points in visualizations (Obeid and Hoque, 2020) and enable users to express their complex information needs about data naturally (Setlur et al, 2016). For example, recent work on Chart Question Answering (QA) has demonstrated how NLP techniques can reduce perceptual and cognitive efforts by automatically answering complex reasoning questions about charts (Kantharaj et al, 2022;Lee et al, 2022) or by generating natural language summaries from charts (Shankar et al, 2022;Obeid and Hoque, 2020). We also refer the interested readers to Prof. Marti Hearst's keynote (link) at IEEE Vis'22 on how NLP can help Visualization.…”

“…• In the tutorial, we will first introduce the domain of NLP+Vis and provide an overview of various downstream tasks in this domain such as question answering with charts (e.g., Lee et al (2022); Kantharaj et al (2022); ), science diagrams (Kembhavi et al, 2016), and infographics (Mathew et al, 2022), as well as natural language generation for visualizations (e.g., Shankar et al (2022)) and text-to-chart (e.g., ).…”

“…His research focuses on combining information visualization and human-computer interaction with natural language processing to address the challenges of the information overload problem. Recently, he has worked on developing natural language interfaces for visualizations (e.g., (Hoque et al, 2017;Setlur et al, 2020)), automatic chart question answering Kantharaj et al, 2022;, chart retrieval (Hoque and Agrawala, 2019) and chart summarization (Shankar et al, 2022;Obeid and Hoque, 2020). He has also worked on developing visual text analytics to support the user's task of exploring and analyzing conversations (e.g., Hoque and Carenini (2014, 2015…”

Progress and Perspective of Artificial Intelligence and Machine Learning of Prediction in Anesthesiology

“…Chart data generation is a crucial step for scaling up the model ability [31,19,2]. Previous chart-related benchmarks only cover general three types of charts (line, pie, bar charts) and focus on a few tasks such as chart-to-table tasks for ChartQA [23], PlotQA [25], and Chart-to-Text [26], and QA tasks for DVQA [13] and OpenCQA [15]. Recently, various benchmarks have been proposed in some works, e.g.…”

SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings

“…This complex task presents a great challenge for the AI community, attracting researchers to develop technologies specifically to address figures. These advancements include AI models that parse information like trends, axes, or statistics in figure images [24,45], generate a variety of descriptions based on the image of the figure (i.e., vision-to-language models) [31,35,37] or the data underlying it (i.e., data-to-text models) [11,41], answer questions about figure images [22,23,38], and suggest appropriate figure types based on the data [20,25,46]. Among these efforts, a significant portion was devoted to the study of scientific figures and their captions in scholarly articles.…”