Doom or Deliciousness: Challenges and Opportunities for Visualization in the Age of Generative Models

Abstract

Generative text-to-image models (as exemplified by DALL-E, MidJourney, and Stable Diffusion) have recently made enormous technological leaps, demonstrating impressive results in many graphical domains—from logo design to digital painting and photographic composition. However, the quality of these results has led to existential crises in some fields of art, leading to questions about the role of human agency in the production of meaning in a graphical context. Such issues are central to visualization, and while these generative models have yet to be widely applied to visualization, it seems only a matter of time until their integration is manifest. Seeking to circumvent similar ponderous dilemmas, we attempt to understand the roles that generative models might play across visualization. We do so by constructing a framework that characterizes what these technologies offer at various stages of the visualization workflow, augmented and analyzed through semi-structured interviews with 21 experts from related domains. Throughthis work, we map the space of opportunities and risks that might arise in this intersection, identifying doomsday prophecies and delicious low-hanging fruits that are ripe for research.

CUPID: Contextual Understanding of Prompt‐conditioned Image Distributions

“…The rise of text‐to‐image generative models [RBL*21, RDN*22, SCS*22] has sparked numerous interface designs [SCK*23, BWS*23, FWW*23, SCM*23] to help humans co‐create with generative models – even for the creation of visualizations [SDBEA*23, WCA23]. Some designs focus on editing a single image at a time [ZRA23, CA23] using methods that either control the sampling process of a diffusion model, or explicitly train a new diffusion model for a target mode of interaction.…”

Generative text-to-image diffusion for automated map production based on geosocial media data

“…Similar to García-Peñalvo & Vázquez-Ingelmo (2023), we use "generative AI" as an umbrella term to describe the emerging class of text-to-text, text-to-image, and image-to-image models. These models have great potential to augment and replace human creativity in many application domains, such as visualization (Schetinger et al 2023) or as an aid to map generation (Juhász et al 2023). Only Juhász explored potential applications of text-to-text models in cartography.…”

“…It has been trained on 5.85 billion images (Schuhmann et al 2022) and published by the CompVis group at LMU Munich and Stability AI (Rombach et al 2022). Schetinger et al (2023) illustrate a number of challenges and opportunities that have emerged. One of the key issues discussed is agency, which is used to describe the ability of analysts to modify the outcome of the content generation process.…”

PMU Tracker: A Visualization Platform for Epicentric Event Propagation Analysis in the Power Grid

“…In the visualization context, aesthetics refers to a quality or characteristic of a visual representation distinct from how clear, informative, or memorable it is. An alternate definition refers to the visual appeal or beauty of the representation [26,37]. We seek to systematically evaluate the applicability of the premise that beauty and functionality are intrinsically intertwined [5,10,22,38,39,41] for visualization.…”