Natural communication with information systems

Toward an affect-sensitive multimodal human-computer interaction

“…Feature-level fusion retains less detailed information than data-level fusion, but it is also less prone to noise and sensor failures, and, most importantly, it is the most appropriate type of fusion for tightly coupled and synchronized modalities. Though many feature-level techniques like Kalman fusion, artificial neural networks (ANN) based fusion, and hidden Markov models (HMM) based fusion have been proposed [26], [105], decision-level (i.e., interpretation-level) fusion is the approach applied most often for multimodal HCI [65], [88], [105]. This practice may follow from experimental studies that that have shown that a late integration approach (i.e., a decision-level fusion) might provide higher recognition scores than an early integration approach (e.g., [100]).…”

“…Though a tactile computer-sensing modality for more natural HCI has been explored recently with increasing interest [65], [77] and although the research in psychophysiology has produced firm evidence that affective arousal has a range of somatic and physiological correlates [12], only a single work aimed at automatic analysis of affective physiological signals has been found in the existing body of literature: the work presented by Picard et al [91]. The lack of interest in this research topic might be in part because of the lack of interest by research sponsors and in part because of the manifold of related theoretical and practical open problems.…”

“…If the goal is to ensure that HCI approaches the naturalness of human-human interaction [65], [105], then the ideal model for multisensory information fusion is the human handling of the multimodal information flow. Insight into how the modalities of sight, sound, and touch are combined in human-human interaction can be gained from neurological studies on fusion of sensory neurons [108].…”

A goal-driven approach for service request modeling

“…In the domain of HCI and HRI, there are several ways to facilitate the context information gathering and user request identification such as visual recognition of hand and body gestures, conversational interaction, force-feedback tactile gloves, or fusing the multimodal input. 9,10 However, in many Web applications, such as the network search engines and Web service systems, a keyboard and a mouse are most general input devices for user-computer interaction.…”

A Physiological Approach to Affective Computing

“…As a result, it is deemed that communication with computers should be more natural and friendlier than the traditional one chiefly relying on hand driven movement using the mouse or the keyboard. Efforts are underway to improve the interface with more intrinsic medium through voice, face expression or gesture and computers are getting human-like (Marsic et al, 2000). Even so it is still far short of what is needed.…”