Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing

Vinciarelli, Alessandro; Pantić, Maja; Heylen, Dirk; Pélachaud, Catherine; Poggi, Isabella; D’Errico, Francesca; Schroeder, Manfred R.

doi:10.1109/t-affc.2011.27

Cited by 322 publications

(255 citation statements)

References 166 publications

Supporting

Mentioning

254

Contrasting

Order By: Relevance

“…Despite being in its initial phase, SSP has already attracted the attention of the technological community: the MIT Technology Review magazine identifies reality mining (one of the main applications of SSP so far), as one of the ten technologies likely to change the world (Greene, 2008), while management experts expect SSP to change organization studies like the microscope has changed medicine a few centuries ago (Buchanan, 2007). What is more important is that the first results in the field attest that social interactions and behaviours, although complex and rooted in the deepest aspects of human psychology, can be analyzed and synthesised automatically with the help of computers (Vinciarelli et al , 2010. However, although fundamental, these are only the first steps, and the journey towards artificial social intelligence and socially-aware computing is still long.…”

Section: Resultsmentioning

confidence: 99%

“…In automatic behaviour synthesis, the aim is thus to create this perception by timely generating suitable signals and behaviours in a synthetic voice, facial expressions and gestures of an Embodied Conversational Agent (ECA). For a comprehensive overview of works on social signal generation on virtual agents, see Vinciarelli et al 2010.…”

Section: Machine Synthesis Of Social Signalsmentioning

confidence: 99%

See 1 more Smart Citation

Social Signal Processing: The Research Agenda

Pantić

Cowie

D’Errico

et al. 2011

Visual Analysis of Humans

Self Cite

View full text Add to dashboard Cite

Abstract. The exploration of how we react to the world and interact with it and each other remains one of the greatest scientific challenges. Latest research trends in cognitive sciences argue that our common view of intelligence is too narrow, ignoring a crucial range of abilities that matter immensely for how people do in life. This range of abilities is called social intelligence and includes the ability to express and recognise social signals produced during social interactions like agreement, politeness, empathy, friendliness, conflict, etc., coupled with the ability to manage them in order to get along well with others while winning their cooperation. Social Signal Processing (SSP) is the new research domain that aims at understanding and modelling social interactions (human-science goals), and at providing computers with similar abilities in human-computer interaction scenarios (technological goals). SSP is in its infancy, and the journey towards artificial social intelligence and socially-aware computing is still long. This research agenda is a twofold, a discussion about how the field is understood by people who are currently active in it and a discussion about issues that the researchers in this formative field face.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Machine Synthesis Of Social Signalsmentioning

confidence: 99%

Social Signal Processing: The Research Agenda

Pantić

Cowie

D’Errico

et al. 2011

Visual Analysis of Humans

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, domains like Affective Computing [31] and Social Signal Processing [43] adopted nonverbal behavioral cues as a physical, machine detectable evidence of emotional and social phenomena, respectively. Research efforts targeted a wide spectrum of problems, including conflict detection [28], communication dynamics [7,25], mimicry measurement [10], early detection of developmental and cognitive diseases [37], role recognition [38], prediction of negotiation outcomes [9], videosurveillance [4,5,6,8], etc.…”

Section: Computingmentioning

confidence: 99%

“…Nonverbal cues influence the perception of social phenomena not only when displayed by humans, but also when synthesized with machines [43]. This makes it possible to create artificial agents that replicate the results of social psychology [18] by interacting with humans like humans do [42].…”

Section: Computingmentioning

confidence: 99%

Predicting online lecture ratings based on gesturing and vocal behavior

Cheng

Salamin

Salvagnini

et al. 2014

J Multimodal User Interfaces

Self Cite

View full text Add to dashboard Cite

Nonverbal behavior plays an important role in any human-human interaction. Teaching -a inherently social activity -is not an exception. So far, the effect of nonverbal behavioral cues accompanying lecture delivery was investigated in the case of traditional ex-cathedra lectures, where students and teachers are co-located. However, it is becoming increasingly more frequent to watch lectures online and, in this new type of setting, it is still unclear what the effect of nonverbal communication is. This article tries to address the problem and proposes experiments performed over the lectures of a popular web repository ("Videolectures"). The results show that automatically extracted nonverbal behavioral cues (prosody, voice quality and gesturing activity) predict the ratings that "Videolectures" users assign to the presentations.

show abstract

“…Although conversational analysis is traditionally a cognitive science endeavor, there is a growing interest in the automatic recognition and synthesis of conversational behavior, particularly for the creation of virtual conversation agents. Recent reviews, [1], [2], provided a detailed overview of this new research field of social signal processing.…”

Section: Introductionmentioning

confidence: 99%

The Face Speaks: Contextual and Temporal Sensitivity to Backchannel Responses

Aubrey

Cunningham

Marshall

et al. 2013

Computer Vision - ACCV 2012 Workshops

View full text Add to dashboard Cite

Abstract. It is often assumed that one person in a conversation is active (the speaker) and the rest passive (the listeners). Conversational analysis has shown, however, that listeners take an active part in the conversation, providing feedback signals that can control conversational flow. The face plays a vital role in these backchannel responses. A deeper understanding of facial backchannel signals is crucial for many applications in social signal processing, including automatic modeling and analysis of conversations, or in the development of life-like, effective conversational agents. Here, we present results from two experiments testing the sensitivity to the context and the timing of backchannel responses. We utilised sequences from a newly recorded database of 5-minute, two-person conversations. Experiment 1 tested how well participants would be able to match backchannel sequences to their corresponding speaker sequence. On average, participants performed well above chance. Experiment 2 tested how sensitive participants would be to temporal misalignments of the backchannel sequence. Interestingly, participants were able to estimate the correct temporal alignment for the sequence pairs. Taken together, our results show that human conversational skills are highly tuned both towards context and temporal alignment, showing the need for accurate modeling of conversations in social signal processing.

show abstract

Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing

Cited by 322 publications

References 166 publications

Social Signal Processing: The Research Agenda

Social Signal Processing: The Research Agenda

Predicting online lecture ratings based on gesturing and vocal behavior

The Face Speaks: Contextual and Temporal Sensitivity to Backchannel Responses

Contact Info

Product

Resources

About