Mutual Learning of an Object Concept and Language Model Based on MLDA and NPYLM

Abstract

SummaryHumans develop their concept of an object by classifying it into a category, and acquire language by interacting with others at the same time. Thus, the meaning of a word can be learnt by connecting the recognized word and concept. We consider such an ability to be important in allowing robots to flexibly develop their knowledge of language and concepts. Accordingly, we propose a method that enables robots to acquire such knowledge. The object concept is formed by classifying multimodal information acquired from objects, and the language model is acquired from human speech describing object features. We propose a stochastic model of language and concepts, and knowledge is learnt by estimating the model parameters. The important point is that language and concepts are interdependent. There is a high probability that the same words will be uttered to objects in the same category. Similarly, objects to which the same words are uttered are highly likely to have the same features. Using this relation, the accuracy of both speech recognition and object classification can be improved by the proposed method. However, it is difficult to directly estimate the parameters of the proposed model, because there are many parameters that are required. Therefore, we approximate the proposed model, and estimate its parameters using a nested PitmanYor language model and multimodal latent Dirichlet allocation to acquire the language and concept, respectively. The experimental results show that the accuracy of speech recognition and object classification is improved by the proposed method.

Inducing Word and Part-of-Speech with Pitman-Yor Hidden Semi-Markov Models

“…NPYLM has already applied and extended to speech recognition (Neubig et al, 2010), statistical machine translation (Nguyen et al, 2010), or even robotics (Nakamura et al, 2014). For all these research area, we believe PYHSMM would be beneficial for their extension.…”

Multimodal Hierarchical Dirichlet Process-Based Active Perception by a Robot

“…Recently, various computational models and machine learning methods for multimodal object categorization have been proposed in artificial intelligence, cognitive robotics, and related research fields (Roy and Pentland, 2002 ; Natale et al, 2004 ; Nakamura et al, 2007 , 2009 , 2011a , b , 2014 ; Iwahashi et al, 2010 ; Sinapov and Stoytchev, 2011 ; Araki et al, 2012 ; Griffith et al, 2012 ; Ando et al, 2013 ; Celikkanat et al, 2014 ; Sinapov et al, 2014 ). For example, Sinapov and Stoytchev ( 2011 ) proposed a graph-based multimodal categorization method that allows a robot to recognize a new object by its similarity to a set of familiar objects.…”

Online Algorithm for Robots to Learn Object Concepts and Language Model

“…First, in a rigorous manner, we formulate the online joint learning of concepts and a language model, based on a generative model. In contrast to this paper, [6] and [7] provide no theoretical formulation of the joint learning problem, and the online algorithm is not involved in [9]. Hence, this is the first attempt to propose an online joint learning algorithm that achieves the aforementioned learning objectives, as shown in Fig.…”

“…Using this relation, the accuracy of both phoneme recognition and object classification can be improved. The idea of joint learning was first proposed in [9] by the co-authors of this paper. However, the algorithm proposed in [9] is a batch-type algorithm, subject to the problems described above.…”

“…The idea of joint learning was first proposed in [9] by the co-authors of this paper. However, the algorithm proposed in [9] is a batch-type algorithm, subject to the problems described above. Therefore, in this paper, the PFoMLDA is extended to a model that can incrementally acquire a language model and the concepts of different objects.…”