Bag of multimodal LDA models for concept formation

Nagaoka, Tomoaki; Nagai, Takayuki; Iwahashi, Naoto

doi:10.1109/icra.2011.5980324

Cited by 32 publications

(32 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy of Estimated Words granularity of categories. Selective attention is the key to model this granularity of categories as discussed in [3]. Another problem to be considered is features of the perceptual information.…”

Section: Inferring Words For Unseen Objectsmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous acquisition of multimodal information for online object concept formation by robots

Araki

Nagaoka

Nagai

et al. 2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

This paper proposes a robot that acquires multimodal information, i.e. auditory, visual, and haptic information, fully autonomous way using its embodiment. We also propose an online algorithm of multimodal categorization based on the acquired multimodal information and words, which are partially given by human users. The proposed framework makes it possible for the robot to learn object concepts naturally in everyday operation in conjunction with a small amount of linguistic information from human users. In order to obtain multimodal information, the robot detects an object on a fla surface. Then the robot grasps and shakes it for gaining haptic and auditory information. For obtaining visual information, the robot uses a hand held small observation table, so that the robot can control the viewpoints for observing the object. As for the multimodal concept formation, the multimodal LDA using Gibbs sampling is extended to the online version in this paper. The proposed algorithms are implemented on a real robot and tested using real everyday objects in order to show validity of the proposed system.

show abstract

Section: Inferring Words For Unseen Objectsmentioning

confidence: 99%

“…An online variational Bayes (VB) algorithm for LDA is shown to be converged to a local optimum of the VB objective function in [13]. We extend Gibbs sampling-based LDA instead of VB-based LDA because Gibbs sampling yields better results in multimodal categorization [3]. Moreover, Gibbs sampling-based LDA has an easy to implement property.…”

Section: Introductionmentioning

confidence: 99%

Autonomous acquisition of multimodal information for online object concept formation by robots

Araki

Nagaoka

Nagai

et al. 2011

2011 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…Rather than a hierarchical structure, these categories, which can be thought of as concepts, have very complex structures. To form such categories, we extended the MLDA to the bag of multimodal LDA (BoMLDA) [7], in which weights for multimodal information are introduced. This model consists of a large number of MLDA models, each of which has a different set of weights for multimodal information.…”

Section: Introductionmentioning

confidence: 99%

Bag of multimodal hierarchical dirichlet processes: Model of complex conceptual structure for intelligent robots

Nagaoka

Nagai

Iwahashi³

2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

The formation of categories, which constitutes the basis of developing concepts, requires multimodal information with a complex structure. We propose a model called the bag of multimodal hierarchical Dirichlet processes (BoMHDP), which enables robots to form a variety of multimodal categories. The BoMHDP model is a collection of a large number of MHDP models, each of which has a different set of weights for sensory information. The weights work to realize selective attention and enable the formation of various types of categories (e.g., object, haptic, and color). The BoMHDP model is an extension of the HDP, and categorization is unsupervised. However, categories that are not natural for humans are also formed. Therefore, only the significant categories are selected through interaction between the user and the robot. At the same time, words obtained during the interaction are connected to the categories. Finally, categories, which are represented by words, are selected. The BoMHDP model was implemented on a robot platform and a preliminary experiment was conducted to validate it. The results revealed that various categories can be formed with the BoMHDP model. We also analyzed the formed conceptual structure by using multidimensional scaling.The results indicate that the complex conceptual structure was represented reasonably well with the BoMHDP model.

show abstract

“…However, they use hand-crafted features with a relatively small number of dimensions. They extend their work in [60] to cope with complex categories structures, for instance when an object belongs to several categories such as toy and soft. To do this, they run their clustering algorithm several times and select relevant clusterings based on correlations with words utterances from the verbal description of objects.…”

Section: Multimodal Fusionmentioning

confidence: 99%

Deep unsupervised network for multimodal perception, representation and classification

Droniou

Ivaldi

Sigaud

2015

Robotics and Autonomous Systems

View full text Add to dashboard Cite

In this paper, we tackle the problem of multimodal learning for autonomous robots. Autonomous robots interacting with humans in an evolving environment need the ability to acquire knowledge from their multiple perceptual channels in an unsupervised way. Most of the approaches in the literature exploit engineered methods to process each perceptual modality. In contrast, robots should be able to acquire their own features from the raw sensors, leveraging the information elicited by interaction with their environment: learning from their sensorimotor experience would result in a more efficient strategy in a life-long perspective. To this end, we propose an architecture based on deep networks, which is used by the humanoid robot iCub to learn a task from multiple perceptual modalities (proprioception, vision, audition). By structuring high-dimensional, multimodal information into a set of distinct sub-manifolds in a fully unsupervised way, it performs a substantial dimensionality reduction by providing both a symbolic representation of data and a fine discrimination between two similar stimuli. Moreover, the proposed network is able to exploit multimodal correlations to improve the representation of each modality alone.

show abstract

Bag of multimodal LDA models for concept formation

Cited by 32 publications

References 10 publications

Autonomous acquisition of multimodal information for online object concept formation by robots

Autonomous acquisition of multimodal information for online object concept formation by robots

Bag of multimodal hierarchical dirichlet processes: Model of complex conceptual structure for intelligent robots

Deep unsupervised network for multimodal perception, representation and classification

Contact Info

Product

Resources

About