Automatic detection of group functional roles in face to face interactions

Abstract

In this paper, we discuss a machine learning approach to automatically detect functional roles played by participants in a face to face interaction. We shortly introduce the coding scheme we used to classify the roles of the group members and the corpus we collected to assess the coding scheme reliability as well as to train statistical systems for automatic recognition of roles. We then discuss a machine learning approach based on multi-class SVM to automatically detect such roles by employing simple features of the visual and acoustical scene. The effectiveness of the classification is better than the chosen baselines and although the results are not yet good enough for a real application, they demonstrate the feasibility of the task of detecting group functional roles in face to face interactions.

Automatic speaker role labeling in AMI meetings: Recognition of formal and social roles

“…Two types of roles have been investigated in different literatures: formal/functional roles, defined from the conversation type, e.g., anchorman/guests in Broadcast data [3,6,5] or professional roles in meetings [7,8], and social roles [12,13,14,15] related to how speakers interact between them. The literature on the first has mainly focused on the use of lexical and structural features while the literature on the second has mainly made use of nonlexical information (prosody and turn-taking statistics).…”

“…On the other other hand, speakers change their social role during the conversation and previous work on social role labeling [12,14] made use of long time windows where the role is considered constant and obtained averaging over several turns. Results reveal (see the summary Table 4) that prosodic features produce the highest recognition (62% correctly labeled time) while the use of structural, lexical and DA information improves the performance up to 64%.…”

“…The amount of speaker's actual speaking time labeled according to the different roles is depicted in Figure 1 where it can be notice that the Protagonist/Supporter roles are the most common one, the Neutral and Gatekeeper are rare roles in those meetings while no instances of Attacker are found because of the collaborative nature of the meetings. As speaker roles continuously change over time, previous works [12,13,14] smoothed the manual annotations considering roles constant over long time-windows (up to one-minute) and assigning speakers the most common role they have in that window thus leveraging a role over several turns. Furthermore these works completely discarded information like words and Dialog Acts considering only prosodic and structural features from the conversation.…”

“…Social roles are useful to characterize the dynamics of the conversation, i.e., the interaction between the participants, they can generalize across any type of conversation and can be related to phenomena studied in meetings like engagement, hot-spots [11] and also social dominance. Previous works on social roles have mainly used non-lexical features [12,13,14] focusing on how participants interact over long time windows (up to one minute) in the conversation, with the exception of [15] where information capturing speaker expressiveness, derived from lexical and prosodic features, was used over short time windows.…”

“…The literature on the first has mainly focused on the use of lexical and structural features [3,5,6,7] while the literature on the second has mainly made use of non-lexical information [12,13,14] (prosody and turn-taking statistics). This work aims at studying in exhaustive way and on the same dataset which of the various features proposed in literature are able to capture information on speaker formal/social roles.…”