An experimental investigation into interaction between language and information graphics in multimodal documents served as the basis for this study. More specifically, our purpose was to investigate the role of linguistic annotations in graph-text documents. Participants were presented with three newspaper articles in the following conditions: one text-only, one text plus non-annotated graph, and one text plus annotated graph. Results of the experiment showed that, on one hand, annotations play a bridging role for integration of information contributed by different representational modalities. On the other hand, linguistic annotations have negative effects on recall, possibly due to attention divided by the different parts of a document.
Natural language as a modality of interaction is becoming increasingly popular in the field of visualization. In addition to the popular query interfaces, other language-based interactions such as annotations, recommendations, explanations, or documentation experience growing interest. In this survey, we provide an overview of natural language-based interaction in the research area of visualization. We discuss a renowned taxonomy of visualization tasks and classify 119 related works to illustrate the stateof-the-art of how current natural language interfaces support their performance. We examine applied NLP methods and discuss humanmachine dialogue structures with a focus on initiative, duration, and communicative functions in recent visualization-oriented dialogue interfaces. Based on this overview, we point out interesting areas for the future application of NLP methods in the field of visualization.
Statistical graphs have been designed for accessible use by visually impaired users. Haptic devices provide an appropriate interface for haptic exploration of statistical graphs. However, haptic exploration of statistical graphs reveals a more local and sequential inspection pattern compared to visual exploration. This difference between haptic exploration and visual exploration is usually attributed to different characteristics of the exploration processes, such as bandwidth of information extraction. To facilitate information extraction from statistical graphs, alternative sensory modalities have been employed. In particular, line graphs have been represented by sound, thus leading to sonified graphs. Despite their demonstrated facilitating effects, sonified graphs have limitations under complex line representations. One method of overcoming those difficulties is to develop a verbal assistance system for haptic line graph comprehension. In the present article, we summarize our studies on designing and developing a verbal assistance system for haptic line graph comprehension. We present the findings in a set of studies conducted with blindfolded and visually impaired participants.
Statistical graphs, such as line graphs are widely used in multimodal communication settings. Language accompanies graphs and humans produce gestures during the course of communication. For visually impaired people, haptic-audio interfaces provide perceptual access to graphical representations. The local and sequential character of haptic perception introduces limitations in haptic perception of hard-to-encode information, which can be resolved by providing audio assistance. In this article we first present a review of multimodal interactions between gesture, language and graphical representations. We then focus on methodologies for investigating hard-toencode information in graph comprehension. Finally, we present a case study to provide insight for designing audio assistance.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.