Complex conversational scene analysis using wearable sensors

Hung, Hayley; Gedik, Ekin; Quiros, Laura Cabrera

doi:10.1016/b978-0-12-814601-9.00019-5

Cited by 4 publications

(4 citation statements)

References 53 publications

(74 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, moving from controlled laboratory studies to reallife settings requires a fundamental change in experimental approaches. As argued by Hung et al, 11 we need to transition from expecting clearly visible video footage of frontal faces and use other sensing modalities to exploit the arsenal of social signals that are emitted by humans.…”

Section: Recent Work On An Agreement Framework Proves To Support Humamentioning

confidence: 99%

A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence

et al. 2020

Self Cite

View full text Add to dashboard Cite

Section: Recent Work On An Agreement Framework Proves To Support Humamentioning

confidence: 99%

A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence

et al. 2020

Self Cite

View full text Add to dashboard Cite

“…We first establish the importance of the interaction context that we are interested in, which is different from the ones in previous studies and their interpretations. While human interactions in focused settings [5,39,44], e.g., seated meetings (Figure 1(a)), have been studied extensively, a closer analysis of complex conversational scenes [27], e.g., networking events or cocktail parties [57] (Figure 1(b) and 1(c)), is more challenging. We differentiate interacting groups in complex conversational scenes from free-standing conversations group (FCG) which have been studied in the past [57].…”

Section: :2 • Tan Et Almentioning

confidence: 99%

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Tan

Tax

Hung

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

Self Cite

View full text Add to dashboard Cite

Human head orientation estimation has been of interest because head orientation serves as a cue to directed social attention. Most existing approaches rely on visual and high-fidelity sensor inputs and deep learning strategies that do not consider the social context of unstructured and crowded mingling scenarios. We show that alternative inputs, like speaking status, body location, orientation, and acceleration contribute towards head orientation estimation. These are especially useful in crowded and in-the-wild settings where visual features are either uninformative due to occlusions or prohibitive to acquire due to physical space limitations and concerns of ecological validity. We argue that head orientation estimation in such social settings needs to account for the physically evolving interaction space formed by all the individuals in the group. To this end, we propose an LSTM-based head orientation estimation method that combines the hidden representations of the group members. Our framework jointly predicts head orientations of all group members and is applicable to groups of different sizes. We explain the contribution of different modalities to model performance in head orientation estimation. The proposed model outperforms baseline methods that do not explicitly consider the group context, and generalizes to an unseen dataset from a different social event.

show abstract

“…However, the issues highlighted in this tutorial are crucial to consider when collecting data in semi-public spaces. The tutorial is partially based on the book chapter by Hung et al [7] providing more detailed practical advice at all levels of the collection process. In addition, themes more specific the data collection process of ConfLab will also be discussed.…”

Section: Course Descriptionmentioning

confidence: 99%

Multimodal Data Collection for Social Interaction Analysis In-the-Wild

Hung

Raman

Gedik

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

The benefits of exploiting multi-modality in the analysis of human-human social behaviour has been demonstrated widely in the community. An important aspect of this problem is the collection of data-sets that provide a rich and realistic representation of how people actually socialize with each other in real life. These subtle coordination patterns are influenced by individual beliefs, goals, and, desires related to what an individual stands to lose or gain in the activities they perform in their every day life. These conditions cannot be easily replicated in a lab setting and require a radical rethinking of both how and what to collect. This tutorial provides a guide on how to create such multi-modal multi-sensor data sets when holistically considering the entire experimental design and data collection process. CCS CONCEPTS • Hardware → Sensor applications and deployments; Wireless integrated network sensors; • Information systems → Social networks; • Human-centered computing Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

Complex conversational scene analysis using wearable sensors

Cited by 4 publications

References 53 publications

A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence

A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Multimodal Data Collection for Social Interaction Analysis In-the-Wild

Contact Info

Product

Resources

About