Task constraints on robot realism: the case of talking heads

Vatikiotis‐Bateson, Eric; Kroos, Christian; Kuratate, Takaaki; Munhall, Kevin G.; Pitermann, Michel

doi:10.1109/roman.2000.892522

Cited by 9 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in the case of Mandarin tone 3 (low-dipping in terms of height and contour), the correlated head/neck motion during tone production should be signaled by a low-fallingrising movement. When present, these visual cues seem to be used by listeners during auditory-visual perception (Vatikiotis-Bateson, Kroos, Kuratate, Munhall, & Pitermann, 2000), which has been shown by our finding of a significant higher accuracy in the auditory-visual condition as compared to the audio-only condition. Such result indicates that visual information helps tone-naïve participants to identify Mandarin tones.…”

Section: Discussionsupporting

confidence: 76%

Relative Contribution of Auditory and Visual Information to Mandarin Chinese Tone Identification by Native and Tone-naïve Listeners

et al. 2019

View full text Add to dashboard Cite

Speech perception is a multisensory process: what we hear can be affected by what we see. For instance, the McGurk effect occurs when auditory speech is presented in synchrony with discrepant visual information. A large number of studies have targeted the McGurk effect at the segmental level of speech (mainly consonant perception), which tends to be visually salient (lip-reading based), while the present study aims to extend the existing body of literature to the suprasegmental level, that is, investigating a McGurk effect for the identification of tones in Mandarin Chinese. Previous studies have shown that visual information does play a role in Chinese tone perception, and that the different tones correlate with variable movements of the head and neck. We constructed various tone combinations of congruent and incongruent auditory-visual materials (10 syllables with 16 tone combinations each) and presented them to native speakers of Mandarin Chinese and speakers of tone-naïve languages. In line with our previous work, we found that tone identification varies with individual tones, with tone 3 (the low-dipping tone) being the easiest one to identify, whereas tone 4 (the high-falling tone) was the most difficult one. We found that both groups of participants mainly relied on auditory input (instead of visual input), and that the auditory reliance for Chinese subjects was even stronger. The results did not show evidence for auditory-visual integration among native participants, while visual information is helpful for tone-naïve participants. However, even for this group, visual information only marginally increases the accuracy in the tone identification task, and this increase depends on the tone in question.

show abstract

Section: Discussionsupporting

confidence: 76%

Relative Contribution of Auditory and Visual Information to Mandarin Chinese Tone Identification by Native and Tone-naïve Listeners

et al. 2019

View full text Add to dashboard Cite

show abstract

“…It is known, in fact, that there is a very high correlation between the dynamic of the vocal tract and the behavior of the facial movement, as emphasized by Yehia et al in Reference [24]. This was used in Reference [25] to develop the natural animation of a talking head. In addition, as regards the talking mechanical robot, if a mechanical vocal stretch is embedded in an artificial head that emulates a human head, the artificial head should have natural movements during the production of speech spoken by the robot, provided that the artificial head is tied to the articulators by means of a kind of elastic joint.…”

Section: Motivationsmentioning

confidence: 99%

An Effective and Efficient Genetic-Fuzzy Algorithm for Supporting Advanced Human-Machine Interfaces in Big Data Settings

2019

View full text Add to dashboard Cite

In this paper we describe a novel algorithm, inspired by the mirror neuron discovery, to support automatic learning oriented to advanced man-machine interfaces. The algorithm introduces several points of innovation, based on complex metrics of similarity that involve different characteristics of the entire learning process. In more detail, the proposed approach deals with an humanoid robot algorithm suited for automatic vocalization acquisition from a human tutor. The learned vocalization can be used to multi-modal reproduction of speech, as the articulatory and acoustic parameters that compose the vocalization database can be used to synthesize unrestricted speech utterances and reproduce the articulatory and facial movements of the humanoid talking face automatically synchronized. The algorithm uses fuzzy articulatory rules, which describe transitions between phonemes derived from the International Phonetic Alphabet (IPA), to allow simpler adaptation to different languages, and genetic optimization of the membership degrees. Large experimental evaluation and analysis of the proposed algorithm on synthetic and real data sets confirms the benefits of our proposal. Indeed, experimental results show that the vocalization acquired respects the basic phonetic rules of Italian languages and that subjective results show the effectiveness of multi-modal speech production with automatic synchronization between facial movements and speech emissions. The algorithm has been applied to a virtual speaking face but it may also be used in mechanical vocalization systems as well.

show abstract

“…Presenting all of the above stimuli (30,2000, and 45 images) requires too much time for subjects, so we reduced the 2000 low quality image set to a reasonable amount by randomly selecting a smaller set of stimuli, ensuring that the same number of images from each morph percentile were represented. We thus used 185 images for the female mask experiment, and 65 for the male and average mask experiments in the low quality group.…”

Section: Stimuli Synthesismentioning

confidence: 99%

Effects of 3D Shape and Texture on Gender Identification for a Retro-Projected Face Screen

Kuratate

Riley

Cheng

2013

Int J of Soc Robotics

View full text Add to dashboard Cite

Retro-projected face displays have recently appeared as an alternative to mechanical robot faces, and stand apart by virtue of their flexibility: they are able to present a variety of faces varying in both realism and individual appearance. Here we examine the role of both 3D mask structure and texture image quality on the perception of gender in one such platform, the Mask-bot. In our experiments, we use three specific gender face screens as the 3D outputfemale, male and average masks -and display various face images that are gradually morphed between female and male on these screens. Additionally, we present three cases of morphed images: high quality texture, low quality texture, and averaged face texture from low quality data. Experiments were carried out over several days. 15 subjects rated the gender of each face projected on the female mask screen, and 10 subjects rated the gender of faces on the male and average screens. We found that even though the 3D mask screens have strong gender specific face features, gender identification is strongly determined by high-quality texture images. However, in the absence of strong texture cues or the presence of ambiguous information, the influence of the output structure may become more important. These results allow us to ascertain the ability to faithfully represent faces on these new platforms, and highlight the most important aspects -in this case texture -for correct perception.

show abstract

Task constraints on robot realism: the case of talking heads

Cited by 9 publications

References 16 publications

Relative Contribution of Auditory and Visual Information to Mandarin Chinese Tone Identification by Native and Tone-naïve Listeners

Relative Contribution of Auditory and Visual Information to Mandarin Chinese Tone Identification by Native and Tone-naïve Listeners

An Effective and Efficient Genetic-Fuzzy Algorithm for Supporting Advanced Human-Machine Interfaces in Big Data Settings

Effects of 3D Shape and Texture on Gender Identification for a Retro-Projected Face Screen

Contact Info

Product

Resources

About