On Cognitive Relevance in Automatic Multimodal Systems

Elouazizi, Noureddine; Bachvarova, Yulia

doi:10.1109/mmse.2004.54

Cited by 7 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SYSTEM: Ok, please wait. [The context engine generates a situated document compliant with the context and the user choice (10) and visualizes it on the user device (11).] USER: .…”

Section: An Example Of Situated Document Processingmentioning

confidence: 99%

Situated Multimodal Documents

Celentano¹,

Pittarello²

2007

18th International Conference on Database and Expert Systems Applications (DEXA 2007)

View full text Add to dashboard Cite

The choices made by user in processing a set of documents is related, in a broad sense, to the sum of influences coming from the documents in the user situation, which does not include only physical or technical context parameters, but also the user goal and a set of personal components related to the user goal and to the effect on the user of the document We use the term situated document to denote such influence. In this paper we propose a model for situated documents and propose a functional architecture for their management

show abstract

“…SYSTEM: Ok, please wait. [The context engine generates a situated document compliant with the context and the user choice (10) and visualizes it on the user device (11).] USER: .…”

Section: An Example Of Situated Document Processingmentioning

confidence: 99%

Situated Multimodal Documents

Celentano¹,

Pittarello²

2007

18th International Conference on Database and Expert Systems Applications (DEXA 2007)

View full text Add to dashboard Cite

show abstract

“…In [10] adaptation is approached with a different viewpoint. Assuming that different combinations of media could convey the same meaning to the user, the authors present adaptation as the selection and combination of media which have the greatest cognitive impact on the user, according to psychology studies about the human perception.…”

Section: Related Workmentioning

confidence: 99%

Context-aware design of adaptable multimodal documents

Celentano

Gaggi

2006

Multimed Tools Appl

View full text Add to dashboard Cite

In this paper we present a model and an adaptation architecture for context-aware multimodal documents. A compound virtual document describes the different ways in which multimodal information can be structured and presented. Physical features are associated to media instances, while properties describe the context. Concrete documents are instantiated from virtual documents by selecting and synchronizing proper media instances based on the user context: the situation, the environment, the device and the available communication resources. The relations between the context features and the media properties are described by a rule based system.

show abstract

“…Based on this knowledge, automatic modality planning can be conducted in a cognitive-aware manner, which means that the human cognitive resources are efficiently used while a user is perceiving and processing the multimodal messages. It has also been argued in [12] that human-computer interaction (interfacing) should be understood in the light of a general principle of human cognition.…”

Section: Introductionmentioning

confidence: 99%

Modality effects on cognitive load and performance in high-load information presentation

Cao

Theune

Nijholt

2009

Proceedings of the 14th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

In this study, we argue that modality planning in multimodal presentation systems needs to consider the modality characteristics at not only the presentational level but also the cognitive level, especially in a situation where the information load is high and the user task is time-critical. As a first step towards automatic cognitive-aware modality planning, we integrated the effect of different modalities on cognitive load and performance, using a high-load information presentation scenario. Mainly based on modality-related psychology theories, we selected five modality conditions (text, image, text+image, text+speech, and text+sound) and made hypotheses about their effects on cognitive load. Modality effects were evaluated by two cognitive load measurements and two performance measurements. Results confirmed most of the predicted modality effects, and showed that these effects become significant when the information load and the task demand are high. The findings of this study suggest that it is highly necessary to encode modality-related principles of human cognition into the modality planning procedure for systems that support high-load human-computer interaction.

show abstract

On Cognitive Relevance in Automatic Multimodal Systems

Cited by 7 publications

References 11 publications

Situated Multimodal Documents

Situated Multimodal Documents

Context-aware design of adaptable multimodal documents

Modality effects on cognitive load and performance in high-load information presentation

Contact Info

Product

Resources

About