Temporal encoded F-formation system for social interaction detection

Abstract

International audienceIn the context of a social gathering, such as a cocktail party, the memorable moments are generally captured by professional photographers or by the participants. The latter case is often undesirable because many participants would rather enjoy the event instead of being occupied by the photo-taking task. Motivated by this scenario, we propose the use of a set of cameras to automatically take photos. Instead of performing dense analysis on all cameras for photo capturing, we first detect the occurrence and location of social interactions via F-formation detection. In the sociology literature, F-formation is a concept used to define social interactions, where each detection only requires the spatial location and orientation of each participant. This information can be robustly obtained with additional Kinect depth sensors. In this paper, we propose an extended F-formation system for robust detection of interactions and interactants. The extended F-formation system employs a heat-map based feature representation for each individual, namely Interaction Space (IS), to model their location, orientation, and temporal information. Using the temporally encoded IS for each detected interactant, we propose a best-view camera selection framework to detect the corresponding best view camera for each detected social interaction. The extended F-formation system is evaluated with synthetic data on multiple scenarios. To demonstrate the effectiveness of the proposed system, we conducted a user study to compare our best view camera ranking with human's ranking using real-world dat

Social interaction detection using a multi-sensor approach

“…We have proposed a heat map based F-formation representation as the social attribute to model the social interaction, which has been accepted for publication at ACM MM 2013 [5]. Specifically, we model the F-formation with temporal information by creating a temporal-encoded heat map.…”

Group Activity Recognition with Group Interaction Zone Based on Relative Distance Between Human Objects

“…They analyzed the interaction of people considering the con¯guration of objects around the bubble zone of proxemics in every time step. Gan et al 11 proposed an extended F-formation based on a heat map 21 to take temporal information into consideration to¯nd groups suitable for a photo shot. In a similar manner, Cui et al 10 proposed Interaction Energy Potential to detect normal and abnormal events based on the Social Force Model 15 which models pedestrian's behavior.…”

Recovering Social Interaction Spatial Structure from Multiple First-Person Views

“…In prior work, social interaction has been studied with static third-person view cameras (e.g., surveillance cameras, Kinect depth sensors, etc.) [1,2,5,9]. However, the static nature of these cameras restricts their usage and potential to recover the full spatial information of a social interaction.…”