Modeling tone and intonation in Mandarin and English as a process of target approximation

Abstract

This paper reports the development of a quantitative target approximation (qTA) model for generating F(0) contours of speech. The qTA model simulates the production of tone and intonation as a process of syllable-synchronized sequential target approximation [Xu, Y. (2005). "Speech melody as articulatorily implemented communicative functions," Speech Commun. 46, 220-251]. It adopts a set of biomechanical and linguistic assumptions about the mechanisms of speech production. The communicative functions directly modeled are lexical tone in Mandarin and lexical stress in English and focus in both languages. The qTA model is evaluated by extracting function-specific model parameters from natural speech via supervised learning (automatic analysis by synthesis) and comparing the F(0) contours generated with the extracted parameters to those of natural utterances through numerical evaluation and perceptual testing. The F(0) contours generated by the qTA model with the learned parameters were very close to the natural contours in terms of root mean square error, rate of human identification of tone, and focus and judgment of naturalness by human listeners. The results demonstrate that the qTA model is both an effective tool for research on tone and intonation and a potentially effective system for automatic synthesis of tone and intonation.

Toward invariant functional representations of variable surface fundamental frequency contours: Synthesizing speech melody via model-based stochastic learning

“…This modeling effort is a significant extension of our previous modeling work, including Prom-on et al (2009) and the subsequently developed PENTAtrainer1 Prom-on, 2010-2013). In the following two subsections we will describe the new components added to the previous system.…”

“…Testing such predictive powers would not only help demonstrate theories' generalizability, but also make them readily applicable to speech technology once the test results are positive. The present study is part of our continued effort in this direction, with a significant extension from our previous work (Prom-on et al, 2009), and with particular focus on the problem of variability. Before describing our current work, however, we will first discuss the main sources of prosodic variability and review how they have been addressed so far.…”

“…Pitch targets represent the goals of F 0 control and are localized to the host syllables (demarcated by the boundaries represented by the vertical gray lines). This conceptual model has been mathematically implemented as the quantitative Target Approximation (qTA) model (Prom-on et al, 2009). In qTA, for each syllable, F 0 is represented by the solution equation of the third-order critically damped linear system driven by a pitch target, as shown in the following equation,…”

“…1A also demonstrates that contextual variability is anything but trivial. In fact, much effort has been devoted to the understanding and modeling of this variability in terms of tonal coarticulation (Gu et al, 2007;Kochanski and Shih, 2003;Prom-on et al, 2009;Shen, 1990;Shih, 1987;Wu, 1984). However, theories and models of intonation rarely address the issue of contextual tonal variations explicitly in their original frameworks (Beckman and Pierrehumbert, 1986;Ladd, 2008;Pierrehumbert, 1980;Taylor, 2000;'t Hart et al, 1990).…”

“…Among the various aspects of prosody, fundamental frequency (F 0 ) is by far the most challenging, and has attracted most of the research effort. Many theories and computational models of F 0 patterns have been proposed over the years (Anderson et al, 1984;Bailly and Holm, 2005;Black and Hunt, 1996;Fujisaki et al, 2005;Grabe et al, 2007;Hirst, 2005Hirst, , 2011Jilka et al, 1999;Kochanski and Shih, 2003;Mixdorff et al, 2003;Pierrehumbert, 1980Pierrehumbert, , 1981Prom-on et al, 2009;Taylor, 2000;van Santen and Möbius, 2000;Xu and Wang, 2001;Xu, 2005), and a large number of empirical studies have been conducted (as reviewed by Wagner and Watson, 2010;ShattuckHufnagel and Turk, 1996;Xu, 2011). Despite the extensive effort, however, most of the critical issues still remain unresolved and some are still under heated debate (Arvaniti and Ladd, 2009;Ladd, 2008;Wagner and Watson, 2010;Wightman, 2002;Xu, 2011).…”