Double Articulation Analyzer With Prosody for Unsupervised Word and Phone Discovery

Abstract

Word and phone discovery are important tasks in the language development of human infants. Infants acquire words and phones from unsegmented speech signals using segmentation cues such as distributional, prosodic, and cooccurrence information. Many pre-existing computational models designed to represent this process tend to focus on distributional or prosodic cues. In this study, we propose a nonparametric Bayesian probabilistic generative model called the prosodic hierarchical Dirichlet process-hidden language model (prosodic HDP-HLM) designed to perform simultaneous phone and word discovery from continuous speech signals encoded as timeseries data that may exhibit a double articulation structure. Prosodic HDP-HLM, as an extension of HDP-HLM, considers both prosodic and distributional cues within a single integrative generative model. We further propose a prosodic double articulation analyzer (Prosodic DAA) based on an inference procedure derived for prosodic HDP-HLM. We conducted three experiments on different types of datasets, including, Japanese vowel sequence, utterances for teaching object names and features, and utterances following Zipf's law, and the results demonstrated the validity of the proposed method. The results show that the Prosodic DAA successfully used prosodic cues and was able to discover words directly from continuous human speech using distributional and prosodic information in an unsupervised manner, outperforming a method that solely used distributional cues. In contrast, the phone discovery performance did not improve. We also show that prosodic cues contributed to word discovery performance more when the word frequency was distributed more naturally, i.e., following Zipf's law.

Unsupervised Multimodal Word Discovery Based on Double Articulation Analysis With Co-Occurrence Cues

“…Our future study prospects include (i) enabling word discovery that incorporates prosodic cues [56] into the proposed method, and (ii) using the acquired words for human-robot interaction and feedback learning through speech synthesis.…”

“…The two different types of word dictionaries are prepared; a generic word dictionary is a builtin large-scale word dictionary, and a true word dictionary consists only of the words in the dataset. These results were taken as reference values from Experiment 2 in Okuda et al's paper [56], where a similar dataset is used.…”