The MatchNMingle dataset A novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates

We investigate the task of detecting speakers in crowded environments using a single body worn triaxial accelerometer. Detection of such behaviour is very challenging to model as people's body movements during speech vary greatly. Similar to previous studies, by assuming that body movements are indicative of speech, we show experimentally, on a real-world dataset of 3 h including 18 people, that transductive parameter transfer learning (Zen et al. in Proceedings of the 16th international conference on multimodal interaction. ACM, 2014) can better model individual differences in speaking behaviour, significantly improving on the state-of-the-art performance. We also discuss the challenges introduced by the in-thewild nature of our dataset and experimentally show how they affect detection performance. We strengthen the need for an adaptive approach by comparing the speech detection problem to a more traditional activity (i.e. walking). We provide an analysis of the transfer by considering different source sets which provides a deeper investigation of the nature of both speech and body movements, in the context of transfer learning.

This paper focuses on the automatic classification of selfassessed personality traits from the HEXACO inventory during crowded mingle scenarios. We exploit acceleration and proximity data from a wearable device hung around the neck. Unlike most state-of-the-art studies, addressing personality estimation during mingle scenarios provides a challenging social context as people interact dynamically and freely in a face-to-face setting. While many former studies use audio to extract speech-related features, we present a novel method of extracting an individual's speaking status from a single body worn triaxial accelerometer which scales easily to large populations. Moreover, by fusing both speech and movement energy related cues from just acceleration, our experimental results show improvements on the estimation of Humility over features extracted from a single behavioral modality. We validated our method on 71 participants where we obtained an accuracy of 69% for Honesty, Conscientiousness and Openness to Experience. To our knowledge, this is the largest validation of personality estimation carried out in such a social context with simple wearable sensors. CCS Concepts •Computing methodologies → Supervised learning by classification; Transfer learning; •Human-centered computing → Ubiquitous and mobile computing design and evaluation methods;

In this paper, we propose a method for detecting conversing groups. More specifically, we detect pairwise F-formation membership using a single worn accelerometer. We focus on crowded real life scenarios, specifically mingling events, where groups of different sizes naturally occur and evolve over time. Our method uses the dynamics of interaction, derived from people's coordinated social actions and movements. The social actions, speaking, head and hand gesturing, are inferred from wearable acceleration with a transfer learning approach. These automatically labeled actions, together with the raw acceleration, are used to define joint representations of interaction between people through the extraction of pairwise features. We present a new feature set based on the overlap patterns of social actions and utilize some others that were previously proposed in other domains. Our approach considers various interaction patterns of different sized groups by training multiple classifiers with respect to cardinality. The final estimation is then dynamically performed by meta-classifier learning using the local neighborhood of the current test sample. We experimentally show that the proposed method outperforms state of the art approaches. Finally, we show how the accuracy of the social action detection affects group detection performance, analyze the effectiveness of features for different group sizes in detail, discuss how different types of features contribute to the final performance and evaluate the effects of using the local neighborhood for meta-classifier learning.

No abstract