A fusion framework for multimodal interactive applications

Mendonça, Hildeberto; Lawson, Jean-Yves Lionel; Vybornova, Olga; Macq, Benoı̂t; Vanderdonckt, Jean

doi:10.1145/1647314.1647344

Cited by 5 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SKEMMI implements computer vision techniques for mass detection, feature extraction and clustering, and incorporates augmented reality, multimodal high-level fusion [10], and multiple video projections. Mass clustering decomposes the input signals of the mass to form clusters (e.g., corresponding to a portion of the mass) to interpret their signals more accurately.…”

Section: Skemmi: a Development Environment For Mass-computer Interactionmentioning

confidence: 99%

“…Users produce individual signals that need to be fused by mass clustering, while the output should be subjected to multi-level fission [10] at the individual, cluster, and mass levels of granularity. SKEMMI displays individual signals via a pulsing aura and shows scores at the level of a user, a cluster of users (e.g., a region), a group (e.g., a team in an auditorium), and the mass, which promotes social recognition and stimulation in competition.…”

Section: Mass-computer Interaction Characteristicsmentioning

confidence: 99%

See 1 more Smart Citation

Mass-Computer Interaction for Thousands of Users and Beyond

Lawson¹,

Vanderdonckt

Vatavu

2018

Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

We introduce Mass-Computer Interaction (MCI) as a natural evolution of Crowd-Computer Interaction (CCI) fostered by recent technical innovations and advances in large-scale sensing, processing, and interactive systems. MCI represents a sensible combination of (1) a very large number of end-users, usually in the order of hundreds or thousands, (2) very large physical settings, such as theaters and auditoriums, and (3) large-scale infrastructure, including distributed systems. We outline design challenges posed by the new MassComputer Interaction paradigm, elaborate on its defining characteristics, and provide a general-purpose model for MCI applications. These contributions are exemplified with SKEMMI, our general-purpose platform specifically designed for developing and deploying Mass-Computer Interaction applications. Mass-Computer Interaction for Thousands of Users and Beyond AbstractWe introduce Mass-Computer Interaction (MCI) as a natural evolution of Crowd-Computer Interaction (CCI) fostered by recent technical innovations and advances in large-scale sensing, processing, and interactive systems. MCI represents a sensible combination of (1) a very large number of end-users, usually in the order of hundreds or thousands, (2) very large physical settings, such as theaters and auditoriums, and (3) large-scale infrastructure, including distributed systems. We outline design challenges posed by the new Mass-Computer Interaction paradigm, elaborate on its defining characteristics, and provide a general-purpose model for MCI applications. These contributions are exemplified with SKEMMI, our general-purpose platform specifically designed for developing and deploying Mass-Computer Interaction applications.

show abstract

Section: Skemmi: a Development Environment For Mass-computer Interactionmentioning

confidence: 99%

Section: Mass-computer Interaction Characteristicsmentioning

confidence: 99%

Mass-Computer Interaction for Thousands of Users and Beyond

Lawson¹,

Vanderdonckt

Vatavu

2018

Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The key reasoning behind its use is that human interaction is multimodal in the real world [74,75], including interactions between an individual and trainer [76]. The order and pattern by which multimodal information should be integrated has long been the subject of study in the field of human-computer interactions [77][78][79][80][81]. To discuss the various ways by which multimodal feedback can be presented, the 2×2 classification of multimodal interfaces by Nigay and Coutaz [82] is used as a basis (Table 1).…”

Section: Multimodal Integrationmentioning

confidence: 99%

A Survey of Multimodal Systems and Techniques for Motor Learning

Tadayon

McDaniel²,

Panchanathan

2017

J Inf Process Syst

View full text Add to dashboard Cite

This survey paper explores the application of multimodal feedback in automated systems for motor learning. In this paper, we review the findings shown in recent studies in this field using rehabilitation and various motor training scenarios as context. We discuss popular feedback delivery and sensing mechanisms for motion capture and processing in terms of requirements, benefits, and limitations. The selection of modalities is presented via our having reviewed the best-practice approaches for each modality relative to motor task complexity with example implementations in recent work. We summarize the advantages and disadvantages of several approaches for integrating modalities in terms of fusion and frequency of feedback during motor tasks. Finally, we review the limitations of perceptual bandwidth and provide an evaluation of the information transfer for each modality.

show abstract