Relating Articulatory Motions in Different Speaking Rates

Abstract

Movements of articulators (e.g., tongue, lips and jaw) in different speaking rates are related in a complex manner. In this work, we examine the underlying function to transform articulatory movements involved in producing speech at a neutral speaking rate into those at fast and slow speaking rates (N2F and N2S). For this we use articulatory movement data collected from five subjects using an Electromagnetic articulograph at neutral, fast and slow speaking rates. As candidate transformation functions (TF), we use affine transformations with a diagonal matrix and a full matrix and a nonlinear function modeled by a deep neural network (DNN). Since the duration of an utterance in different speaking rates would typically be unequal, it is required to time align the articulatory movement trajectories, which, in turn, affects the TF learnt. Therefore, we propose an iterative algorithm to alternately optimize for the TF and the time alignments. Subject specific experiments reveal that while N2F transformation can be well described by an affine transformation with a full matrix, N2S transformation is better represented by a more complex nonlinear function modeled by a DNN. This could be because subjects exhibit gross articulatory movements during fast speech and hyper-articulate while producing slow speech.

Attention and Encoder-Decoder Based Models for Transforming Articulatory Movements at Different Speaking Rates

“…The experimental setup and data-set for this work are similar to the preliminary work [1]. The proposed AstNet model outperforms the technique proposed in [1] in both cases (N2F and N2S) for all subjects. DTW distance is used as an evaluation metric, with lower distances indicating better performance.…”

“…The transformation should be able to learn the variations in duration and range of movements of articulators from source speaking rate to target speaking rate. Preliminary work has been done on this topic [1], where variations in the range of articulatory movements are modeled using various transformation methods like an affine transformation with both diagonal and full matrix and a non-linear transformation modeled by a deep neural network (DNN). As all these transformation methods learn work at the frame level, which require dynamic time warping (DTW) to time-align articulatory movement trajectories at different rates.…”

“…The limitations of this approach lies in modelling the duration variations across speaking rates. To overcome these shortcomings in [1], the proposed approach utilizes attention mechanism to capture the duration variations to improve the performance. The proposed encoderdecoder based attention network used to perform Articulatory trajectories Speaking rate Transformation is denoted as AstNet.…”

“…AstNet is an end-to-end model which takes articulatory movements at one rate as input and generates at a different rate. This is in contrast with the iterative approach followed in [1]. In addition, we don't need to time-align prior to training the model as the attention network learns the duration modelling and by using "stop token" method, end of sequence is predicted at the time of inference.…”

“…

While speaking at different rates, articulators (like tongue, lips) tend to move differently and the enunciations are also of different durations. In the past, affine transformation and DNN have been used to transform articulatory movements from neutral to fast(N2F) and neutral to slow(N2S) speaking rates [1]. In this work, we improve over the existing transformation techniques by modeling rate specific durations and their transformation using AstNet, an encoder-decoder framework with attention.

…”