Symbol emergence in robotics: a survey

Taniguchi, Tadahiro; Nagai, Takayuki; Nagaoka, Tomoaki; Iwahashi, Naoto; Ogata, Tetsuya; Asoh, Hideki

doi:10.1080/01691864.2016.1164622

Cited by 117 publications

(85 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More importantly, these results imply that the concept and language learning processes can be viewed as emergent systems [44]. At early stages of learning, concepts and the language model are built in a bottom-up manner and, after a while, work as constraints to correctly recognize teaching utterances, which refines concepts and the language model.…”

Section: Discussionmentioning

confidence: 92%

Online Algorithm for Robots to Learn Object Concepts and Language Model

Nishihara

Nagaoka

Nagai

2017

IEEE Trans. Cogn. Dev. Syst.

Self Cite

View full text Add to dashboard Cite

Humans form concept of objects by classifying them into categories, and acquire language by simultaneously interacting with others. Thus, the meaning of a word can be learned by connecting a recognized word to its corresponding concept. We consider this ability important for robots to flexibly develop knowledge of language and concepts. In this paper, we propose an online algorithm for robots to acquire knowledge of natural language and learn object concepts. A robot learns the language model from word sequences, which are obtained by the segmentation of phoneme sequences provided by a user, by using unsupervised word segmentation each time it is provided with a new object. Moreover, the robot acquires object concepts using these word sequences as well as multimodal information obtained by observing objects. The crucial aspect of our model is the interdependence of words and concepts: there is a high probability that the same words will be uttered to describe objects in the same category. By taking this relationship into account, our proposed method enables robots to acquire a more accurate language model and object concepts online. Experimental results verify this.

show abstract

Section: Discussionmentioning

confidence: 92%

Online Algorithm for Robots to Learn Object Concepts and Language Model

Nishihara

Nagaoka

Nagai

2017

IEEE Trans. Cogn. Dev. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Pareto front of multiple criteria will then has to be considered using additional method provided by the mlrMBO R-package [34], [35]. Another possible benefit of our framework is that the exploration and storing of different optimized set of hyper-parameters can lead to embodied symbols or concepts emergence [36]. By providing human labels about the physical aspect of objects (e.g.…”

Section: Discussionmentioning

confidence: 99%

Developmental Bayesian Optimization of Black-Box with Visual Similarity-Based Transfer Learning

Petit

Depierre

Wang

et al. 2018

2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

View full text Add to dashboard Cite

We present a developmental framework based on a long-term memory and reasoning mechanisms (Vision Similarity and Bayesian Optimisation). This architecture allows a robot to optimize autonomously hyper-parameters that need to be tuned from any action and/or vision module, treated as a black-box. The learning can take advantage of past experiences (stored in the episodic and procedural memories) in order to warmstart the exploration using a set of hyper-parameters previously optimized from objects similar to the new unknown one (stored in a semantic memory). As example, the system has been used to optimized 9 continuous hyper-parameters of a professional software (Kamido) both in simulation and with a real robot (industrial robotic arm Fanuc) with a total of 13 different objects. The robot is able to find a good object-specific optimization in 68 (simulation) or 40 (real) trials. In simulation, we demonstrate the benefit of the transfer learning based on visual similarity, as opposed to an amnesic learning (i.e. learning from scratch all the time). Moreover, with the real robot, we show that the method consistently outperforms the manual optimization from an expert with less than 2 hours of training time to achieve more than 88% of success.

show abstract

“…During subsequent use, a speech query is given and the agent needs to identify which visual object the query refers to. This setting is relevant in modelling infant language acquisition, where models can be used to test particular cognitive hypotheses [5]; lowresource speech processing, where new concepts could be taught in an arbitrary language [6]; and robotics, where novel concepts must be acquired online from co-occurring multimodal sensory inputs [7][8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Multimodal One-shot Learning of Speech and Images

Eloff

Engelbrecht

Kamper

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Imagine a robot is shown new concepts visually together with spoken tags, e.g. "milk", "eggs", "butter". After seeing one paired audiovisual example per class, it is shown a new set of unseen instances of these objects, and asked to pick the "milk". Without receiving any hard labels, could it learn to match the new continuous speech input to the correct visual instance? Although unimodal one-shot learning has been studied, where one labelled example in a single modality is given per class, this example motivates multimodal oneshot learning. Our main contribution is to formally define this task, and to propose several baseline and advanced models. We use a dataset of paired spoken and visual digits to specifically investigate recent advances in Siamese convolutional neural networks. Our best Siamese model achieves twice the accuracy of a nearest neighbour model using pixel-distance over images and dynamic time warping over speech in 11-way cross-modal matching.

show abstract

Symbol emergence in robotics: a survey

Cited by 117 publications

References 77 publications

Online Algorithm for Robots to Learn Object Concepts and Language Model

Online Algorithm for Robots to Learn Object Concepts and Language Model

Developmental Bayesian Optimization of Black-Box with Visual Similarity-Based Transfer Learning

Multimodal One-shot Learning of Speech and Images

Contact Info

Product

Resources

About