Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT-3 Large Language Models

Abstract

The field of data visualisation has long aimed to devise solutions for generating visualisations directly from natural language text. Research in Natural Language Interfaces (NLIs) has contributed towards the development of such techniques. However, the implementation of workable NLIs has always been challenging due to the inherent ambiguity of natural language, as well as in consequence of unclear and poorly written user queries which pose problems for existing language models in discerning user intent. Instead of pursuing the usual path of developing new iterations of language models, this study uniquely proposes leveraging the advancements in pre-trained large language models (LLMs) such as ChatGPT and GPT-3 to convert free-form natural language directly into code for appropriate visualisations. This paper presents a novel system, Chat2VIS, which takes advantage of the capabilities of LLMs and demonstrates how, with effective prompt engineering, the complex problem of language understanding can be solved more efficiently, resulting in simpler and more accurate end-to-end solutions than prior approaches. Chat2VIS shows that LLMs together with the proposed prompts offer a reliable approach to rendering visualisations from natural language queries, even when queries are highly misspecified and underspecified. This solution also presents a significant reduction in costs for the development of NLI systems, while attaining greater visualisation inference abilities compared to traditional NLP approaches that use hand-crafted grammar rules and tailored models. This study also presents how LLM prompts can be constructed in a way that preserves data security and privacy while being generalisable to different datasets. This work compares the performance of GPT-3, Codex and ChatGPT across several case studies and contrasts the performances with prior studies.INDEX TERMS ChatGPT, codex, end-to-end visualisations from natural language, GPT-3, large language models, natural language interfaces, text-to-visualisation.

Bioinformatics Illustrations Decoded by ChatGPT: The Good, The Bad, and The Ugly

“…Data visualization is crucial in conveying results from bioinformatics analyses. Large language model chatbots such as ChatGPT have demonstrated an ability to transform natural language prompts into relevant visual representations through coding 36,37 . The newly introduced feature of ChatGPT to take image inputs offers a promising avenue for identifying patterns within figures, offering interpretations, summarizing findings, and beyond 38 .…”

ChatGPT és más nagy nyelvi modellek (LLM-ek) biztonsági kérdései, szervezeti és társadalmi hatásai

“…Manipuláció: mivel az MI reagál a bemenetre, rosszindulatú támadók manipulálhatják az eredményeket olyan válaszok érdekében, amelyeket ők preferálnak. Egy támadó a rendszert oly módon kérdezi meg, hogy az adott politikai nézeteket vagy hamis információt támogasson (Maddigan 2023).…”

Hyperbolic Personalized Tag Recommendation

“…A typical application is using natural language to generate data visualizations through an interface [45]. Moreover, codes [36] or declarative grammar for visualizations [13], [35], [38] is often used as input or output as a simplified form of code. This type of work demonstrates code comprehension of language models and basic knowledge of visualizations.…”

“…To that end, we leverage LLMs' ability to comprehend the system's information and generate tutorials. Previous studies demonstrate that declarative grammar of visualization is readily understood by language models [35], [36]. To encapsulate automatic tutorial generation, we need a unified specification for different VA systems.…”