A Framework for Analyzing and Designing Diagrams and Graphics

Engelhardt, Yuri; Richards, Clive

doi:10.1007/978-3-319-91376-6_20

Cited by 18 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As our analysis shows, diagrams that represent cycles do not necessarily need to draw on arrows for this purpose: the diagrammatic mode provides alternatives, such as written language, whose structural features (here: cohesive ties) may be used to cue a discourse semantic interpretation involving cyclicity. This allows a fine-grained decomposition of the proposed building blocks of diagrammatic representations [16,10]. Conversely, multimodality research is likely to benefit from the concepts developed in diagrams research for producing systematic descriptions of expressive resources.…”

Section: Discussionmentioning

confidence: 99%

Introducing the diagrammatic semiotic mode

Hiippala¹,

Bateman²

2020

Preprint

View full text Add to dashboard Cite

In this article, we propose a multimodal perspective to diagrammatic representations by sketching a description of what may be tentatively termed the diagrammatic mode. We consider diagrammatic representations in the light of contemporary multimodality theory and explicate what enables diagrammatic representations to integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations.

show abstract

Section: Discussionmentioning

confidence: 99%

Introducing the diagrammatic semiotic mode

Hiippala¹,

Bateman²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We proceed to use the guidance in Fig. 1 of [5] to apply the "types of visual encoding" to our domain. A collection of diagrams can be found at aidiagrams.com/resources, on which this more general commentary is based.…”

Section: Mode Of Correspondencementioning

confidence: 99%

“…The framework proposed by Engelhardt and Richards [5] is designed to be applicable to "all types of visualizations". Their framework provides a method and vocabulary for analysing diagrams and diagram components, which they have tested on a large variety of visualization types.…”

Section: Introductionmentioning

confidence: 99%

Understanding Scholarly Neural Network System Diagrams Through Application of VisDNA

Marshall

Jay

Freitas

2021

Diagrammatic Representation and Inference

View full text Add to dashboard Cite

We utilise Richards-Engelhardt framework as a tool for understanding Natural Language Processing systems diagrams. Through four examples from scholarly proceedings, we find that the application of the framework to this ecological and complex domain is effective for reflecting on these diagrams. We argue for vocabulary to describe multiplecodings, semiotic variability, and inconsistency or misuse of visual encoding principles in diagrams. Further, for application to scholarly Natural Language Processing systems, and perhaps systems diagrams more broadly, we propose the addition of "Grouping by Object" as a new visual encoding principle, and "Emphasising" as a new visual encoding type.

show abstract

“…Due to their widespread use, diagrams have been studied from various perspectives. Previous research has examined their visual perception (Hegarty and Just 1993;Ware 2012), structure and functions (Engelhardt 2002;Purchase 2014;Engelhardt and Richards 2018) and their role as a tool for supporting thinking and reasoning (Tversky 2015) and use in education and instruction (Tippett 2016), to name but a few examples.…”

Section: Introductionmentioning

confidence: 99%

AI2D-RST: a multimodal corpus of 1000 primary school science diagrams

Hiippala

Alikhani

Haverinen

et al. 2020

Lang Resources & Evaluation

View full text Add to dashboard Cite

This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology. The corpus is based on the Allen Institute for Artificial Intelligence Diagrams (AI2D) dataset, a collection of diagrams with crowdsourced descriptions, which was originally developed to support research on automatic diagram understanding and visual question answering. Building on the segmentation of diagram layouts in AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that provides a rich description of their multimodal structure. Annotated by trained experts, the layers describe (1) the grouping of diagram elements into perceptual units, (2) the connections set up by diagrammatic elements such as arrows and lines, and (3) the discourse relations between diagram elements, which are described using Rhetorical Structure Theory (RST). Each annotation layer in AI2D-RST is represented using a graph. The corpus is freely available for research and teaching.

show abstract

A Framework for Analyzing and Designing Diagrams and Graphics

Cited by 18 publications

References 7 publications

Introducing the diagrammatic semiotic mode

Introducing the diagrammatic semiotic mode

Understanding Scholarly Neural Network System Diagrams Through Application of VisDNA

AI2D-RST: a multimodal corpus of 1000 primary school science diagrams

Contact Info

Product

Resources

About