The MatchNMingle Dataset: A Novel Multi-Sensor Resource for the Analysis of Social Interactions and Group Dynamics In-the-Wild During Free-Standing Conversations and Speed Dates

Cabrera-Quirós, Laura; Demetriou, Andrew M.; Gedik, Ekin; Meij, Leander van der; Hung, Hayley

doi:10.1109/taffc.2018.2848914

Cited by 51 publications

(77 citation statements)

References 62 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our experiments, we used the data from the MatchN-Mingle dataset which includes data from three separate speed dating events took place in a local bar and joined by 92 participants in total [15]. In each event, speed dating sessions were followed by a free mingling session of roughly one hour.…”

Section: Datamentioning

confidence: 99%

“…During these 30 minutes, groups of various sizes ranging from dyadic interactions to seven people were formed, creating a rich testing environment. For more details regarding the dataset, please refer to [15].…”

Section: Datamentioning

confidence: 99%

“…Since we use a pairwise representation, this results in three different classes for each sample from pairs that were already in an interaction: speaker&speaker, speaker&listener and listener&listener. We use the speaking status ground truth present in the MatchNMingle [15] to label someone as a speaker or a listener. If a person is annotated as speaking at least 30% of the window that the sample is extracted from, this person is labelled as a speaker and listener otherwise.…”

Section: Joint Prediction Of Membership and Rolesmentioning

confidence: 99%

See 2 more Smart Citations

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Rosatelli

Gedik

Hung

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the use of proxemics and dynamics for automatically identifying conversing groups, or so-called F-formations. More formally we aim to automatically identify whether wearable sensor data coming from 2 people is indicative of F-formation membership. We also explore the problem of jointly detecting membership and more descriptive information about the pair relating to the role they take in the conversation (i.e. speaker or listener). We jointly model the concepts of proxemics and dynamics using binary proximity and acceleration obtained through a single wearable sensor per person. We test our approaches on the publicly available MatchNMingle dataset which was collected during real-life mingling events. We find out that fusion of these two modalities performs significantly better than them independently, providing an AUC of 0.975 when data from 30-second windows are used. Furthermore, our investigation into roles detection shows that each role pair requires a different time resolution for accurate detection.

show abstract

Section: Datamentioning

confidence: 99%

Section: Datamentioning

confidence: 99%

Section: Joint Prediction Of Membership and Rolesmentioning

confidence: 99%

See 1 more Smart Citation

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Rosatelli

Gedik

Hung

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For our experiments, we use the MatchNMingle dataset [5],where it was collected video and wearable acceleration for 92 participants during 3 separated group gatherings. This data was collected in a real mingling scenario after a speed date event, where people were encouraged to socialize.…”

Section: Real Mingling Scenario Datasetmentioning

confidence: 99%

“…We hypothesize that such confusions are due to the role of social context and in this section we intend to analyze this aspect further. To do so, we used social actions annotations provided with the MatchNMingle dataset [5] and specify on Section 3. From all 8 social actions provided, we focus in hand gestures and speaking which are more related to conversational aspects of the context.…”

Section: Association Vs Social Contextmentioning

confidence: 99%

A Hierarchical Approach for Associating Body-Worn Sensors to Video Regions in Crowded Mingling Scenarios

Cabrera-Quirós

Hung

2019

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

We address the complex problem of associating several wearable devices with the spatio-temporal region of their wearers in video during crowded mingling events using only acceleration and proximity. This is a particularly important first step for multi-sensor behavior analysis using video and wearable technologies, where the privacy of the participants must be maintained. Most state-of-the-art works using these two modalities perform their association manually, which becomes practically unfeasible as the number of people in the scene increases. We proposed an automatic association method based on a hierarchical linear assignment optimization, which exploits the spatial context of the scene. Moreover, we present extensive experiments on matching from 2 to more than 69 acceleration and video streams, showing significant improvements over a random baseline in a real world crowded mingling scenario. We also show the effectiveness of our method for incomplete or missing streams (up to a certain limit) and analyze the trade-off between length of the streams and number of participants. Finally, we provide an analysis of failure cases, showing that deep understanding of the social actions within the context of the event is necessary to further improve performance on this intriguing task.

show abstract

Mining Exceptional Social Behaviour

Jorge

Atzmueller

Heravi

et al. 2019

Progress in Artificial Intelligence

View full text Add to dashboard Cite

Essentially, our lives are made of social interactions. These can be recorded through personal gadgets as well as sensors adequately attached to people for research purposes. In particular, such sensors may record real time location of people. This location data can then be used to infer interactions, which may be translated into behavioural patterns. In this paper, we focus on the automatic discovery of exceptional social behaviour from spatio-temporal data. For that, we propose a method for Exceptional Behaviour Discovery (EBD). The proposed method combines Subgroup Discovery and Network Science techniques for finding social behaviour that deviates from the norm. In particular, it transforms movement and demographic data into attributed social interaction networks, and returns descriptive subgroups. We applied the proposed method on two real datasets containing location data from children playing in the school playground. Our results indicate that this is a valid approach which is able to obtain meaningful knowledge from the data.

show abstract

The MatchNMingle Dataset: A Novel Multi-Sensor Resource for the Analysis of Social Interactions and Group Dynamics In-the-Wild During Free-Standing Conversations and Speed Dates

Abstract: The MatchNMingle dataset A novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates

Cited by 51 publications

References 62 publications

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

A Hierarchical Approach for Associating Body-Worn Sensors to Video Regions in Crowded Mingling Scenarios

Mining Exceptional Social Behaviour

Contact Info

Product

Resources

About