A genetic-fuzzy algorithm for the articulatory imitation of facial movements during vocalization of a humanoid robot

Abstract

Abstract-In human heads there is a strong structural linkage between vocal tract and facial behavior during speech. For a robotic talking head to have a human-like behavior, this linkage should be emulated. One way to do that is to compute an estimate of the articulatory features which produce a given utterance and then to transform them into facial animation. We present a computational model of human vocalization which is aimed at describing the articulatory mechanisms which produce spoken phonemes. It uses a set of fuzzy rules and genetic optimization. The former represents the relationships between places of articulations and speech acoustic parameters, while the latter estimates the degrees of membership of the places of articulation. That is, the places of articulation are considered as fuzzy sets whose degrees of membership are the articulatory features. The trajectories of articulatory parameters can be used to control a graphical or mechanical talking head. We verify the model presented here by generating and listening to artificial sentences. Subjective listening tests of artificially generated sentences from the articulatory description resulted in an average phonetic accuracy of about 79 %. Through the analysis of a large amount of natural speech, the algorithm can be used to learn the places of articulation of all phonemes of a given speaker.

Improved humanoid vocalization acquisition from a human tutor

“…The digital model leads to important results in the generation of artificial vocalization both in terms of speech and facial movements. This paper make a breakthrough over the algorithms described in [4] [5]. The major points of improvement, which are the contributions of this paper, are the following:…”

“…Our algorithm uses the imitation learning approach proposed in [4] for the Italian language. In this model, articulatory parameters are estimated from the tutor speech using a genetic optimization scheme.…”

“…• while the rules adopted in [4][5] were tailored to the Italian language, the rules developed in this paper use the International Phonetic Alphabet (IPA) [6], thus leading to the possibility to extend the system to other languages; • while in [4][5] we used single words and short phrases, we describe in this paper a system able to produce vocalization from unrestricted text; • the algorithm uses the average pitch extracted from the tutor as a base of the artificial prosody; • we include in this paper extensive subjective evaluations of the proposed system that were impossible to perform previously. The unrestricted text vocalization is performed by acquiring, in an initial training phase, a data base of small speech units suitably defined.…”

An Interactive Receptionist Robot for Users with Vocal Pathologies

“…The algorithm we use in this work for virtual talking head is described in greater details in [30]. In the rest of this section we give a brief description of the algorithm for the sake of completeness.…”

An Effective and Efficient Genetic-Fuzzy Algorithm for Supporting Advanced Human-Machine Interfaces in Big Data Settings

“…• while the rules in Reference [32] refer to the Italian language, the rules developed in this paper use the International Phonetic Alphabet (IPA) [44] thus enabling the system to be extended to other languages; • while in Reference [32] we use single words and short phrases, in this paper we describe a system that produces unrestricted vocalizations; • while in References [32,45] we control only four points around the lip, that is four face muscles to produce facial movements, in this paper we extend the number of controlled muscles to the left and right cheek, thus leading to better facial movements; • the algorithm described here uses the average pitch extracted from the tutor as the basis of the artificial prosody; • finally, we include in this paper extensive subjective evaluations of the proposed system that were impossible to execute earlier.…”