Emotional speech: Towards a new generation of databases

Abstract

Research on speech and emotion is moving from a period of exploratory research into one where there is a prospect of substantial applications, notably in human-computer interaction. Progress in the area relies heavily on the development of appropriate databases. This paper addresses four main issues that need to be considered in developing databases of emotional speech: scope, naturalness, context and descriptors. The state of the art is reviewed. A good deal has been done to address the key issues, but there is still a long way to go. The paper shows how the challenge of developing appropriate databases is being addressed in three major recent projects--the Reading-Leeds project, the Belfast project and the CREST-ESP project. From these and other studies the paper draws together the tools and methods that have been developed, addresses the problems that arise and indicates the future directions for the development of emotional speech databases. Ó 2002 Elsevier Science B.V. All rights reserved.Re esume e LÕe etude de la parole et de lÕe emotion, partie du stade de la recherche exploratrice, en arrive maintenant au stade qui est celui dÕapplications importantes, notamment dans lÕinteraction homme-machine. Le progre es en ce domaine de epend e etroitment du de eveloppement de bases de donne ees approprie ees. Cet article aborde quatre points principaux qui me eritent notre attention a a ce sujet: lÕe etendue, lÕauthenticite e, le contexte et les termes de description. Il pre esente un compte-rendu de la situation actuelle dans ce domaine et e evoque les avance ees faites, et celles qui restent a a faire. LÕarticle montre comment trois re ecents projets importants (celui de Reading-Leeds, celui de Belfast, et celui de CREST-ESP) ont releve e le de efi pose e par la construction de bases de donne ees approprie ees. A partir de ces trois projets, ainsi que dÕautres travaux, les auteurs pre esentment un bilan des outils et me ethodes utilise es, identifient les proble emes qui y sont associe es, et indiquent la direction dans laquelle devraient sÕorienter les recherches a a venir.

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

“…Generally, few acted-emotion speech databases have included speakers with several different native languages. More recently, some attempts to collect multimodal data were made: some examples of multimodal databases can be found in [17], [18], [19].…”

Describing the emotional states that are expressed in speech

“…There is still limited data on the prevalence and nature of emotional speech in natural settings, but pointers can be drawn from recent collections of natural emotional speech, the Leeds-Reading database and the Belfast database (Douglas- Cowie et al, 2003).…”

“…For example, the database collected by the Reading group (Douglas-Cowie et al, 2003) describes an utterance as moving from ''hate'' to ''vengeful anger''. That approach has the advantage that it does not throw away very much information.…”

“…They proposed two levels of descriptor, one characterising the antecedent events associated with the emotion, the other describing valenced responses to it (e.g., Ôreproach emotionÕ, indicating disapproval of another personÕs actions). The system was applied to the Leeds-Reading database (Douglas- Cowie et al, 2003), but there is currently no detailed description of its characteristics.…”

Naturalistic emotional speech collection paradigm with online game and its psychological and acoustical assessment

“…However, it is too stylized to synthesize delicate nuances of expressions. Naturalistic emotional speech enables one to synthesize more natural and genuine emotional speech that conveys delicate nuances of human emotional expression [7]. Automatic speech recognition (ASR) is also another example that requires a naturalistic emotional speech database.…”