Analyzing, modelling, and specifying visual interaction

Abstract

This paper discusses the processes underlying human-computer visual interaction, thereby analysing the characteristics of visual interaction; a model and a theory of visual interaction, from which a formal specification of visual interactive systems that are trustable by their users can be derived, are also illustrated. Such a theory is called theory of visual sentences, since each message on the computer screen is described as a visual sentence, i.e., an element of a visual language that specifies the interaction. The concept of relational structure is introduced to take into account different kinds of relations which can exist among the characteristic patterns present in a visual sentence. A formal model of the dynamics of visual interaction is presented, which is specified as transformations of visual sentences, modelled through visual rewriting systems, and transformations of relational structures. Particular attention is given to usability issues so as to satisfy relevant features needed to allow non-ambiguity of interpretation, adequate communication, determinism and system viability. An example of visual interaction is given where an immunologist interacts with a simulation of the human immune system

Visual interaction: models, systems, prototypes. The Pictorial Computing Laboratory at the University of Rome “La Sapienza”

“…The impact on these definitions on several phenomena related to interaction and on design and usability of visual interactive systems has been discussed in the light of this approach (Bottoni et al 1999c(Bottoni et al , 2000a(Bottoni et al , 2002b. In particular, it has been shown how interaction reaches a high computational power, leading to phenomena of undecidability (Bottoni et al 1998e), how cognitive pitfalls can be avoided by defining suitable orders on visual sentences (Bottoni et al 1998a(Bottoni et al , 1999b, or expressed as derived from frame effects (Bottoni et al 2002c), or how to use formal specifications to derive models of dialogue control (Bottoni et al 1997b(Bottoni et al , 1998b(Bottoni et al , 1999a, and of the dynamics of visual interaction (Bottoni et al 1998c(Bottoni et al , 2002a.…”

Languages, tasks and roles: modeling interaction through resources.

“…Semantic errors are related to the double process of interpretation occurring in visual interaction [3]. Indeed, while a computational system maintains a diagram description d C associated with the graphical structures in a picture ι, a user operates according to a private description d U for the same ι, possibly using a different set of graphical structures.…”