Trajectory learning and reproduction for differential drive mobile robots based on GMM/HMM and dynamic time warping using learning from demonstration framework

Vuković, Najdan; Mitić, Marko; Miljković, Zoran

doi:10.1016/j.engappai.2015.07.002

Cited by 24 publications

(10 citation statements)

References 32 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the ground truth of the training data as a hidden variable is an effective method to estimate the parameters of PGM [36]. Hence, the Hidden Markov model(HMM) is proposed to deal with the hidden parameters system, now it has been well applied in many control and decision making system, such as robot control [37]- [40], autonomous manufacturing [41]- [43], fault diagnosis [44], [45].…”

Section: Related Workmentioning

confidence: 99%

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Yang

Lin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Applying model-based learning for the optimal decision of the multi-agent system is not trivial due to the expensive price or even the impossibility of obtaining the ground truth for training the model of the complex environment. Such as learning the optimal action of hydraulic supports in the top-coal caving, the optimal action could not accessible as the ground truth of the corresponding state in the intricate processes. Regarding the latent ground truth as the hidden variable is an effective method in the hidden Markov model. This paper extends the hidden variable of ground truth to the multi-agent system and proposes the hidden Markov random field (HMRF) model with reinforcement learning for optimizing the action decision of the multi-agent. In the HMRF model, the input states and the output actions of the multi-agent are considered as an observable random field and a latent Markov random field, respectively. Based on the HMRF model, the optimal decision is inferred by the maximum posterior probability with the prior probability obtained by Q-learning. Meanwhile, the parameters of the HMRF model are estimated by the expectation maximum algorithm. In the experiment, the top-coal caving demonstrates the effectiveness of the proposed method that the recall of top-coal is improved prominently with a very small price of increasing the rock-rate. Furthermore, the proposed method is employed to deal with the predator-preys problem in the gym. The experiment result shows that the communication between agents by the HMRF increases the reward of the preys. INDEX TERMS Hidden Markov random field, optimal decision, multi-agent, top-coal caving.

show abstract

Section: Related Workmentioning

confidence: 99%

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Yang

Lin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, it is not always clear that the underlying true reward, in the sense of being the unique reward an expert may have used, is re-constructable or even if it can be sufficiently approximated. Combining multiple demonstrations to blend a desired expert response as in Vukoviundefined et al [36] may not recreate an expected output with divergent multi-clustered demonstrations, which we are interested in the current work. Alternatively, Angelov et al [1] and Gombolay et al [14] propose a solution that is based on composing smaller policies to mitigate the search for hierarchical decomposition of the demonstration through direct learning of a goal scoring metric or through pair-wise ranking.…”

Section: Learning From Demonstrationmentioning

confidence: 99%

From demonstrations to task-space specifications. Using causal analysis to extract rule parameterization from demonstrations

Angelov

Hristov

Ramamoorthy

2020

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

Learning models of user behaviour is an important problem that is broadly applicable across many application domains requiring human–robot interaction. In this work, we show that it is possible to learn generative models for distinct user behavioural types, extracted from human demonstrations, by enforcing clustering of preferred task solutions within the latent space. We use these models to differentiate between user types and to find cases with overlapping solutions. Moreover, we can alter an initially guessed solution to satisfy the preferences that constitute a particular user type by backpropagating through the learned differentiable models. An advantage of structuring generative models in this way is that we can extract causal relationships between symbols that might form part of the user’s specification of the task, as manifested in the demonstrations. We further parameterize these specifications through constraint optimization in order to find a safety envelope under which motion planning can be performed. We show that the proposed method is capable of correctly distinguishing between three user types, who differ in degrees of cautiousness in their motion, while performing the task of moving objects with a kinesthetically driven robot in a tabletop environment. Our method successfully identifies the correct type, within the specified time, in 99% [97.8–99.8] of the cases, which outperforms an IRL baseline. We also show that our proposed method correctly changes a default trajectory to one satisfying a particular user specification even with unseen objects. The resulting trajectory is shown to be directly implementable on a PR2 humanoid robot completing the same task.

show abstract

“…[17] first combined the output probability of the normalised neural network with the transmission probability of the HMM model. In addition to its application in speech recognition, GMM-HMM models are used in capturing remarkable changes in the state of the mobile robot's motion [26], and segmenting human activities.…”

Section: Gmm-hmm Modelmentioning

confidence: 99%

Individual Commute Time Recognition Based on the Hierarchical Semantic Model

et al. 2020

View full text Add to dashboard Cite

Individual commute time recognition is essential for traffic demand management. However, this problem has yet to be studied. In this study, we propose a hierarchical semantic model (HSM) to recognize individual commute time. To the best of our knowledge, this work is the first to integrates large scale travellers commute time prediction at an individual level. HSM consists of a low and a high semantic layer. The low semantic layer models spatial, temporal and environmental information, whereas the high semantic layer recognises commute time using the hidden Markov model on the basis of the low semantic layer outputs. Experimental results demonstrate the effectiveness of our proposed model for individual commute time recognition. INDEX TERMS commute time recognition, hierarchical semantic model, traffic demand management

show abstract

Trajectory learning and reproduction for differential drive mobile robots based on GMM/HMM and dynamic time warping using learning from demonstration framework

Cited by 24 publications

References 32 publications

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

From demonstrations to task-space specifications. Using causal analysis to extract rule parameterization from demonstrations

Individual Commute Time Recognition Based on the Hierarchical Semantic Model

Contact Info

Product

Resources

About