Emotional head motion predicting from prosodic and linguistic features

Yang, Minghao; Jiang, Jinlin; Tao, Jianhua; Mu, Kaihui; Li, Hao

doi:10.1007/s11042-016-3405-3

Cited by 5 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These features would allow the preparation of parameters that can improve the classification efficiency of viseme groups consisting of phonemes with a strong involvement of the tongue during the articulation of a given speech fragment. In this context, one could consider using data from a specialized device of electromagnetic articulography with its adequate parametrization or as shown in a recent paper by Yang et al [43], one can also to employ emotional head motion predicting from prosodic and linguistic features or data acquired from a face motion capture device [24].…”

Section: Conclusion and Directions For Further Researchmentioning

confidence: 99%

A comparative study of English viseme recognition methods and algorithms

Jachimski

Czyżewski

Ciszewski

2017

Multimed Tools Appl

View full text Add to dashboard Cite

An elementary visual unit -the viseme is concerned in the paper in the context of preparing the feature vector as a main visual input component of Audio-Visual Speech Recognition systems. The aim of the presented research is a review of various approaches to the problem, the implementation of algorithms proposed in the literature and a comparative research on their effectiveness. In the course of the study an optimal feature vector construction and an appropriate selection of the classifier were sought. The experimental research was conducted on the basis of a spoken corpus in which speech was represented both acoustically and visually. The extracted features represented three types: geometrical, textural and mixed ones. The features were processed employing the classification algorithms based on Hidden Markov Models and Sequential Minimal Optimization. Tests were carried out employing the processed video material recorded with English native speakers who read specially prepared list of commands. The obtained results are discussed in the paper.

show abstract

Section: Conclusion and Directions For Further Researchmentioning

confidence: 99%

A comparative study of English viseme recognition methods and algorithms

Jachimski

Czyżewski

Ciszewski

2017

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…Three-dimensional head and face motion and the acoustics of a talker producing Japanese sentences were recorded and analyzed previously in this context (Munhall et al 2004). The inherent relationship by building the mapping model between head motions and Chinese speech prosody and linguistic features has been also studied more recently (Minghao et al 2016).…”

Section: Historical Viewmentioning

confidence: 99%

Database of speech and facial expressions recorded with optimized face motion capture settings

Kawaler

Czyżewski

2019

J Intell Inf Syst

View full text Add to dashboard Cite

The broad objective of the present research is the analysis of spoken English employing a multiplicity of modalities. An important stage of this process, discussed in the paper, is creating a database of speech accompanied with facial expressions. Recordings of speakers were made using an advanced system for capturing facial muscle motion. A brief historical outline, current applications, limitations and the ways of capturing face muscle motion as well as the problems with recording facial expressions are discussed. In particular, the scope of the present analysis concerns the registration of facial expressions related to emotions of speakers which accompany articulation. The camera system, instrumentation and software used for registration and for post-production are outlined. An analysis of the registration procedure and the results of the registration process was performed. The obtained results demonstrate how muscle movements can be registered employing reflective markers and point at the advantages and limitations of applying FMC (Face Motion Capture) technology in compiling a multimodal speech database. A short discussion pertaining to the usage of FMC as ground truth data source in facial expression databases concludes the paper.

show abstract

“…Non-verbal cues, e.g., hand gestures and head motions, are used to express feelings, give feedbacks and engage immersive human-human communication. In [14], Yang et al study the relations between speech and head motion and learn a bimodal mapping from speech to head motions. Specifically, they give an interesting investigation to discover what kinds of prosodic and linguistic features have the most significant influence on emotional head motions.…”

Section: Recognizing Humans and Understanding Their Behaviorsmentioning

confidence: 99%