F-Formation Detection: Individuating Free-Standing Conversational Groups in Images

Abstract

Detection of groups of interacting people is a very interesting and useful task in many modern technologies, with application fields spanning from video-surveillance to social robotics. In this paper we first furnish a rigorous definition of group considering the background of the social sciences: this allows us to specify many kinds of group, so far neglected in the Computer Vision literature. On top of this taxonomy we present a detailed state of the art on the group detection algorithms. Then, as a main contribution, we present a brand new method for the automatic detection of groups in still images, which is based on a graph-cuts framework for clustering individuals; in particular, we are able to codify in a computational sense the sociological definition of F-formation, that is very useful to encode a group having only proxemic information: position and orientation of people. We call the proposed method Graph-Cuts for F-formation (GCFF). We show how GCFF definitely outperforms all the state of the art methods in terms of different accuracy measures (some of them are brand new), demonstrating also a strong robustness to noise and versatility in recognizing groups of various cardinality.

F-Formations for Social Interaction in Simulation Using Virtual Agents and Mobile Robotic Telepresence Systems

“…These constraint-based formations are shown in Figure 3. Formations are considered very useful in analyzing and increasing the quality of interaction in social interactions [1,10,11], and a number of works [19][20][21][22][23][24][25] have proposed different methods to detect F-formations automatically. The Hough voting strategy (density estimation) was used to locate the O-space (see Figure 2a) by considering each person's position and head orientation in [19].…”

“…It is an edge-weighted graph model based on body orientation and proximity to find the dominant set [21]. A method called graph-cuts for F-formation (GCFF) for groups in still images using proxemic information was proposed in [22], which introduced a new set of metrics using the idea of a tolerance threshold. In [24], the authors considered body orientation as the primary cue and proposed a joint learning approach to estimate the pose and F-formation for groups in videos.…”

SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

“…We first considered four state-of-the-art visionbased approaches for individuating FCGs in SALSA. Specifically, we adopted (i) Hough voting [30] (HVFFlin), (ii) its non-linear variant [31] and (iii) multi-scale extensions [32] (denoted as HVFF-ent and HVFF-ms) and (iv) the graph cut approach [9] as associated codes are publicly available 4 .…”

Automatic detection of human interactions from RGB-D data for social activity classification

“…These theories have been already exploited for detection of conversational groups on still images. The authors in [13], [14] present an algorithm to detect visually social interactions on RGB images using the concept of F-Formations [4]. The oriented position of the people is exploited to extract a circle (O-Space) through voting of the centre.…”