A Usability Framework for the Design and Evaluation of Multimodal Interaction

Chang, Jaeseung; Bourguet, Marie-Luce

doi:10.4018/978-1-60566-978-6.ch008

Cited by 6 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Chang and Bourguet [3] proposes a practical framework to multimodal interaction design and evaluation based on six different stages: (1) A more theoretical approach is CARE (Complementarity, Assignment, Redundancy, and Equivalence) [5]. Some less generic approaches that arguably take into account only specific cases of multimodality are [4] and [9].…”

Section: Methods For Designing Multimodal Interfacesmentioning

confidence: 99%

“…Finding appropriate modalities that can be combined together in order to execute a certain task appropriately is not trivial [10]. Moreover, since "multimodal systems tend to utilize natural channels of communication, they are widely exposed to human errors and ambiguity" [3]. On the other hand, the use of multiple integrated interaction modes can help mutually disambiguate one another [11], for example when a user points to a certain part of a screen while saying a command that would be fairly ambiguous per se.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Design of communication in multimodal web interfaces

Neto

Fortes

Assis

et al. 2009

Proceedings of the 27th ACM International Conference on Design of Communication

View full text Add to dashboard Cite

In this paper we present our considerations for the design, development and evaluation of Web multimodal interfaces using as example the analysis and the results of three Case Studies. Two usability methods were applied in these Case Studies as part of our approach for Web multimodal interfaces design. The objective is to smooth the progress of the design activities provided by the Multimodal Web Approach (MMWA) recovering Design Rationale from previous projects and improving the usability and user experience in the multimodal interaction. From the experience got performing three case studies with the MMWA, we derive a general framework for design of communication in Web Multimodal Interfaces.

show abstract

Section: Methods For Designing Multimodal Interfacesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Design of communication in multimodal web interfaces

Neto

Fortes

Assis

et al. 2009

Proceedings of the 27th ACM International Conference on Design of Communication

View full text Add to dashboard Cite

show abstract

“…Chang y Bourguet [11] describen la utilidad de las máquinas de estados finitos como una herramienta de modelado para el diseño multimodal, donde se puede describir y entender fácilmente la estructura de las interacciones multimodales usando los conceptos de estado y acciones que provocan cambios de estados. Para el caso de estudio se diseñó la máquina de estados que modela la interacción multimodal a partir de las siguientes acciones (Figura 5):…”

Section: A Integración Multimodalunclassified

Multimodal interaction architecture applied to navigation in maps

Duque

Rosa

Hernández

2013

2013 8th Computing Colombian Conference (8CCC)

View full text Add to dashboard Cite

This paper presents a multimodal interaction architecture that is proposed as the interaction/control component of new or existing computer applications, particularly for 2D/3D visual computing applications. The architecture aims to encourage multimodality in interaction between the user and the computer application in order to achieve a more natural interaction, easier and user friendly. The presented architecture integrates specialized interaction modules in different modalities (hand gestures and voice initially) operating simultaneously. The information resulting from these modules is processed by a multimodal integration module to detect simultaneous actions/commands. The architecture was integrated with BingMaps application to allow navigation in maps through voice and gestures. Some results of an initial evaluation using the current prototype are presented where we get information regarding its functionality and the resulting user experience. Keywords-human computer interaction; user experience; new paradigms of interaction; multimodal interaction; gestual interaction; voice interaction; navigation in maps. I. INTRODUCCIÓNAunque la interacción con el computador se ha basado por décadas en un grupo limitado de modalidades (ratón y teclado), la interacción humana con el medio circundante y con otros seres humanos es sin duda multimodal, es decir que se usan múltiples canales de comunicación para transmitir información (e.g., los gestos manuales, la voz, la escritura, la mirada) o para adquirir información (a través de la vista, el oído y el tacto). El acercar la interacción de los usuarios de una aplicación computacional a mecanismos y metáforas más naturales favorece la usabilidad y la satisfacción para los usuarios. Los recientes avances tecnológicos en las áreas de procesamiento de señales, visión computacional e interacción HombreMáquina han hecho posible el estudio y desarrollo de interfaces multimodales.Este trabajo propone una arquitectura para el desarrollo de interfaces de interacción multimodal, la cual permite que la componente de interacción de una aplicación computacional integre múltiples modalidades, cada modalidad con su procesamiento especializado, para luego integrar sus resultados y definir acciones/comandos conjuntos que se comunican a la aplicación. La expresión de la interacción de la aplicación se define por una máquina de estados finitos donde se identifican los posibles estados de la aplicación y las transiciones entre estados corresponden a las acciones/comandos que debe reconocer la capa de interacción a partir de las entradas dadas por el usuario.La arquitectura propuesta se valida con un caso de estudio en el que se implementa una interfaz de interacción multimodal usando la aplicación Bing-Maps [1] y definiendo un conjunto de gestos manuales y comandos de voz para realizar las acciones comunes en la navegación de mapas. Egenhofer [2] y McGee et al. [3] sugieren que las interfaces multimodales son más eficientes para interactuar con información geo-espacial que la interacción uni...

show abstract

“…Furthermore, it provides a command line interface to the Web-based GUI, which allows for the creation of stand-alone applications outside of the browser window. For the set of uni-and multimodal commands that can be used for I-SEARCH interfaces, the results of Chang [5] as well as the needs derived from the creation of multimodal search queries are used. Figure 5 depicts the internal structure of UIIFace and shows the flow of events.…”

Section: Multimodal Interaction Handling -Uiifacementioning

confidence: 99%

Context-Aware Querying for Multimodal Search Engines

Etzold

Brousseau²,

Grimm

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Multimodal interaction provides the user with multiple modes of interacting with a system, such as gestures, speech, text, video, audio, etc. A multimodal system allows for several distinct means for input and output of data. In this paper, we present our work in the context of the I-SEARCH project, which aims at enabling context-aware querying of a multimodal search framework including real-world data such as user location or temperature. We introduce the concepts of MuSeBag for multimodal query interfaces, UIIFace for multimodal interaction handling, and CoFind for collaborative search as the core components behind the I-SEARCH multimodal user interface, which we evaluate via a user study.

show abstract

A Usability Framework for the Design and Evaluation of Multimodal Interaction

Cited by 6 publications

References 9 publications

Design of communication in multimodal web interfaces

Design of communication in multimodal web interfaces

Multimodal interaction architecture applied to navigation in maps

Context-Aware Querying for Multimodal Search Engines

Contact Info

Product

Resources

About