Online learning of concepts and words using multimodal LDA and hierarchical Pitman-Yor Language Model

Araki, Takaya; Nagaoka, Tomoaki; Nagai, Takayuki; Nagasaka, Shogo; Taniguchi, Tadahiro; Iwahashi, Naoto

doi:10.1109/iros.2012.6385812

Cited by 41 publications

(49 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the method provides the statistical tools for discovering hidden topics in unsupervised data, we propose that it can also be used for modeling context. In fact, ours is not the first attempt to use LDA formulation in robotics: It has been utilized successfully for object categorization from multi-modal sensory data [42]- [44], and for autonomous drive annotation [45]. However, our work is the first to use LDA for modeling context in robotics.…”

Section: A Context In Cognitive Science and Roboticsmentioning

confidence: 97%

Learning Context on a Humanoid Robot using Incremental Latent Dirichlet Allocation

Çelikkanat

Orhan

Pugeault

et al. 2016

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-In this article, we formalize and model context in terms of a set of concepts grounded in the sensorimotor interactions of a robot. The concepts are modeled as a web using Markov Random Field, inspired from the concept web hypothesis for representing concepts in humans. On this concept web, we treat context as a latent variable of Latent Dirichlet Allocation (LDA), which is a widely-used method in computational linguistics for modeling topics in texts. We extend the standard LDA method in order to make it incremental so that (i) it does not re-learn everything from scratch given new interactions (i.e., it is online) and (ii) it can discover and add a new context into its model when necessary. We demonstrate on the iCub platform that, partly owing to modeling context on top of the concept web, our approach is adaptive, online and robust: It is adaptive and online since it can learn and discover a new context from new interactions. It is robust since it is not affected by irrelevant stimuli and it can discover contexts after a few interactions only. Moreover, we show how to use the context learned in such a model for two important tasks: object recognition and planning.

show abstract

Section: A Context In Cognitive Science and Roboticsmentioning

confidence: 97%

Learning Context on a Humanoid Robot using Incremental Latent Dirichlet Allocation

Çelikkanat

Orhan

Pugeault

et al. 2016

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…Robotic studies have investigated situation awareness in urban search for rescue tasks [15], home security [16] and elderly people's living environments [17], object recognition in daily activities [18]. LDA has already been used in robotics; e.g., for learning concepts and their labels [19], or autonomous drive annotation [20]. In computer vision, the notion of context has grown in prominence over the last decade, both explicitly and implicitly.…”

Section: A Context In Cognitive Science and Roboticsmentioning

confidence: 99%

Learning and using context on a humanoid robot using latent dirichlet allocation

Çelikkanat

Orhan

Pugeault

et al. 2014

4th International Conference on Development and Learning and on Epigenetic Robotics

View full text Add to dashboard Cite

Abstract-In this work, we model context in terms of a set of concepts grounded in a robot's sensorimotor interactions with the environment. For this end, we treat context as a latent variable in Latent Dirichlet Allocation, which is widely used in computational linguistics for modeling topics in texts. The flexibility of our approach allows many-to-many relationships between objects and contexts, as well as between scenes and contexts. We use a concept web representation of the perceptions of the robot as a basis for context analysis. The detected contexts of the scene can be used for several cognitive problems. Our results demonstrate that the robot can use learned contexts to improve object recognition and planning.

show abstract

“…Therefore, we compared the proposed method under the following four conditions: (N, L) = (1, 1), (1,20), (10,1), (10,20). (N, L) = (1, 1) is the method proposed in [5].…”

Section: A Word Selectionmentioning

confidence: 99%

“…We believe that such an ability is also important for robots to acquire language flexibly. Therefore, we have applied phoneme recognition without a language model and the nested Pitman-Yor language model (NPYLM) [4] to multimodal categorization [5]. A user's utterances are converted into phoneme sequences; then, these are segmented into words using NPYLM in an unsupervised manner; and finally, the segmented words are connected to concepts formed by multimodal categorization.…”

Section: Introductionmentioning

confidence: 99%

Multimodal concept and word learning using phoneme sequences with errors

Nagaoka

Araki

Nagai

et al. 2013

2013 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

In this study, we propose a method for concept formation and word acquisition for robots. The proposed method is based on multimodal latent Dirichlet allocation (MLDA) and the nested Pitman-Yor language model (NPYLM). A robot obtains haptic, visual, and auditory information by grasping, observing, and shaking an object. At the same time, a user teaches object features to the robot through speech, which is recognized using only acoustic models and transformed into phoneme sequences. As the robot is supposed to have no language model in advance, the recognized phoneme sequences include many phoneme recognition errors. Moreover, the recognized phoneme sequences with errors are segmented into words in an unsupervised manner; however, not all words are necessarily segmented correctly. The words including these errors have a negative effect on the learning of word meanings. To overcome this problem, we propose a method to improve unsupervised word segmentation and to reduce phoneme recognition errors by using multimodal object concepts. In the proposed method, object concepts are used to enhance the accuracy of word segmentation, reduce phoneme recognition errors, and correct words so as to improve the categorization accuracy. We experimentally demonstrate that the proposed method can improve the accuracy of word segmentation and reduce the phoneme recognition error and that the obtained words enhance the categorization accuracy.

show abstract

Online learning of concepts and words using multimodal LDA and hierarchical Pitman-Yor Language Model

Cited by 41 publications

References 8 publications

Learning Context on a Humanoid Robot using Incremental Latent Dirichlet Allocation

Learning Context on a Humanoid Robot using Incremental Latent Dirichlet Allocation

Learning and using context on a humanoid robot using latent dirichlet allocation

Multimodal concept and word learning using phoneme sequences with errors

Contact Info

Product

Resources

About