Enhancing Interpretation of Ambiguous Voice Instructions based on the Environment and the User’s Intention for Improved Human-Friendly Robot Navigation

Muthugala, M. A. Viraj J.; Srimal, P. H. D. Arjuna S.; Jayasekara, A. G. Buddhika P.

doi:10.3390/app7080821

Cited by 10 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fuzzy method [33] Simple model, small dataset requirement Weak generalization ability, predefined rules.…”

Section: Methods Advantages Disadvantagesmentioning

confidence: 99%

“…Some studies try to estimate human intention through various sensors other than vision sensors. Muthugala et al [33] proposed a system to identify the user intention for the interpretation of ambiguous instructions related to distances in robot navigation scenarios. The actual intention is identified by analyzing the gesture information and voice instructions through a rule-based fuzzy method.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model

Zhang

Peng

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

Most robots are programmed to carry out specific tasks routinely with minor variations. However, more and more applications from SMEs require robots work alongside their counterpart human workers. To smooth the collaboration task flow and improve the collaboration efficiency, a better way is to formulate the robot to surmise what kind of assistance a human coworker needs and naturally take the right action at the right time. This paper proposes a prediction-based human-robot collaboration model for assembly scenarios. An embedded learning from demonstration technique enables the robot to understand various task descriptions and customized working preferences. A state-enhanced convolutional long short-term memory (ConvLSTM)-based framework is formulated for extracting the high-level spatiotemporal features from the shared workspace and predicting the future actions to facilitate the fluent task transition. This model allows the robot to adapt itself to predicted human actions and enables proactive assistance during collaboration. We applied our model to the seats assembly experiment for a scale model vehicle and it can obtain a human worker’s intentions, predict a coworker’s future actions, and provide assembly parts correspondingly. It has been verified that the proposed framework yields higher smoothness and shorter idle times, and meets more working styles, compared to the state-of-the-art methods without prediction awareness.

show abstract

“…Fuzzy method [33] Simple model, small dataset requirement Weak generalization ability, predefined rules.…”

Section: Methods Advantages Disadvantagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model

Zhang

Peng

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Fuzzy logic has been proven to be well suited for replicating the human expert knowledge that can be represented through linguistic expressions [32][33][34]. Furthermore, fuzzy logic has the ability to cope with imprecise sensor information [35][36][37].…”

Section: Self-organizing Fuzzy Logic (Sof) Classifiermentioning

confidence: 99%

A Self-Organizing Fuzzy Logic Classifier for Benchmarking Robot-Aided Blasting of Ship Hulls

Muthugala

Cruz

et al. 2020

Sensors

View full text Add to dashboard Cite

Regular dry dock maintenance work on ship hulls is essential for maintaining the efficiency and sustainability of the shipping industry. Hydro blasting is one of the major processes of dry dock maintenance work, where human labor is extensively used. The conventional methods of maintenance work suffer from many shortcomings, and hence robotized solutions have been developed. This paper proposes a novel robotic system that can synthesize a benchmarking map for a previously blasted ship hull. A Self-Organizing Fuzzy logic (SOF) classifier has been developed to benchmark the blasting quality of a ship hull similar to blasting quality categorization done by human experts. Hornbill, a multipurpose inspection and maintenance robot intended for hydro blasting, benchmarking, and painting, has been developed by integrating the proposed SOF classifier. Moreover, an integrated system solution has been developed to improve dry dock maintenance of ship hulls. The proposed SOF classifier can achieve a mean accuracy of 0.9942 with an execution time of 8.42 µs. Realtime experimenting with the proposed robotic system has been conducted on a ship hull. This experiment confirms the ability of the proposed robotic system in synthesizing a benchmarking map that reveals the benchmarking quality of different areas of a previously blasted ship hull. This sort of a benchmarking map would be useful for ensuring the blasting quality as well as performing efficient spot wise reblasting before the painting. Therefore, the proposed robotic system could be utilized for improving the efficiency and quality of hydro blasting work on the ship hull maintenance industry.

show abstract

“…Human prefers to use multiple modalities such as voice and gestures to interact with each other in a domestic environment [6][7][8]. Gestures included hand gestures, facial expressions, or cues which are difficult to understand even for human beings.…”

Section: Introductionmentioning

confidence: 99%

An Intelligent Gesture Classification Model for Domestic Wheelchair Navigation with Gesture Variance Compensation

Bandara

Priyanayana

Jayasekara

et al. 2020

Applied Bionics and Biomechanics

View full text Add to dashboard Cite

Elderly and disabled population is rapidly increasing. It is important to uplift their living standards by improving the confidence towards daily activities. Navigation is an important task, most elderly and disabled people need assistance with. Replacing human assistance with an intelligent system which is capable of assisting human navigation via wheelchair systems is an effective solution. Hand gestures are often used in navigation systems. However, those systems do not possess the capability to accurately identify gesture variances. Therefore, this paper proposes a method to create an intelligent gesture classification system with a gesture model which was built based on human studies for every essential motion in domestic navigation with hand gesture variance compensation capability. Experiments have been carried out to evaluate user remembering and recalling capability and adaptability towards the gesture model. Dynamic Gesture Identification Module (DGIM), Static Gesture Identification Module (SGIM), and Gesture Clarifier (GC) have been introduced in order to identify gesture commands. The proposed system was analyzed for system accuracy and precision using results of the experiments conducted with human users. Accuracy of the intelligent system was determined with the use of confusion matrix. Further, those results were analyzed using Cohen’s kappa analysis in which overall accuracy, misclassification rate, precision, and Cohen’s kappa values were calculated.

show abstract

Enhancing Interpretation of Ambiguous Voice Instructions based on the Environment and the User’s Intention for Improved Human-Friendly Robot Navigation

Cited by 10 publications

References 26 publications

Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model

Prediction-Based Human-Robot Collaboration in Assembly Tasks Using a Learning from Demonstration Model

A Self-Organizing Fuzzy Logic Classifier for Benchmarking Robot-Aided Blasting of Ship Hulls

An Intelligent Gesture Classification Model for Domestic Wheelchair Navigation with Gesture Variance Compensation

Contact Info

Product

Resources

About