User walkthrough of multimodal access to multidimensional databases

Abstract

This paper describes a user walkthrough that was conducted with an experimental multimodal dialogue system to access a multidimensional music database using a simulated mobile device (including a technically challenging four-PHANToM-setup). The main objectives of the user walkthrough were to assess user preferences for certain modalities (speech, graphical and haptictactile) to access and present certain types of information, and for certain search strategies when searching and browsing a multidimensional database. In addition, the project aimed at providing concrete recommendations for the experimental setup, multimodal user interface design and evaluation. The results show that recommendations can be formulated both on the use of modalities and search strategies, and on the experimental setup as a whole, including the user interface. In short, it is found that haptically enhanced buttons are preferred for navigating or selecting and speech is preferred for searching the database for an album or artist. A 'direct' search strategy indicating an album, artist or genre is favorable. It can be concluded that participants were able to look beyond the experimental setup and see the potential of the envisioned mobile device and its modalities. Therefore it was possible to formulate recommendations for future multimodal dialogue systems for multidimensional database access.

Review of Designs for Haptic Data Visualization

“…These are again monosemic objects as they associate one piece of information. Van Esch-Bussemakers and Cremers [79] utilized a complex setup of four PHANTOMs, with their motion restricted to a straight line, along with vibrotactile actuators integrated in the gimbals to simulate a mobile device. The setup was used to experiment how users could access a multidimensional music database.…”

“…While on the other hand, specific modalities may be better at displaying certain data; and be most relevant to a particular task. For instance, van Esch-Bussemakers and Cremers [79] investigated the use of different modalities to operate different functions; concluding "that participants did have preferences for certain modalities when handling certain types of information".…”

Stimulus–response compatibility and affective computing: a review

“…employed multi-modal stimuli carrying an affective load obtaining a multimodal affective congruency effect in several studies (e.g. see Van Esch-Bussemakers 2001, Lemmens et al 2001.…”

“…Lemmens, Bussemakers and colleagues (e.g. see Van Esch-Bussemakers 2001, Lemmens et al 2000) have used the major/minor distinction repeatedly to construct earcons that carry an affective charge in various studies involving a simple picture-categorization task. They found that the positive affective valence of the major mode (expressed as, e.g.…”