Multimodal Natural Human–Computer Interfaces for Computer-Aided Design: A Review Paper

Niu, Hongwei; Leeuwen, Cees van; Hao, Jia; Wang, Guoxin; Lachmann, Thomas

doi:10.3390/app12136510

Cited by 15 publications

(6 citation statements)

References 140 publications

(224 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The IMUs' sensing data is able to be wirelessly transmitted via Bluetooth. [42,43] The sensing data acquired from each IMU encompassed three-axis acceleration data as well as three-axis angular velocity data. [44] Through an internal data processing algorithm, these data were transformed into quaternions.…”

Section: Teaching Robots Using the Wearable Dataglovementioning

confidence: 99%

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

Zou,

Liu,

Zhao

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Efficient object handover between humans and robots holds significant importance within collaborative manufacturing environments. Enhancing the efficacy of human–robot handovers involves enabling robots to comprehend and foresee human handover intentions. This article introduces human‐teaching–robot‐learning‐prediction framework, allowing robots to learn from diverse human demonstrations and anticipate human handover intentions. The framework facilitates human programming of robots through demonstrations utilizing augmented reality and a wearable dataglove, aligned with task requirements and human working preferences. Subsequently, robots enhance their cognitive capabilities by assimilating insights from human handover demonstrations, utilizing deep neural network algorithms. Furthermore, robots can proactively seek clarification from humans via an augmented reality system when confronted with ambiguity in human intentions, mirroring how humans seek clarity from their counterparts. This proactive approach empowers robots to anticipate human intentions and assist human partners during handovers. Empirical results underscore the benefits of the proposed approach, demonstrating highly accurate prediction of human intentions in human–robot handover tasks.

show abstract

Section: Teaching Robots Using the Wearable Dataglovementioning

confidence: 99%

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

Zou,

Liu,

Zhao

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…The traditional single input output mode human-machine interface can no longer meet people's needs, so multi-mode natural human-machine interface has emerged. Niu et al [13] analyzed that new human-machine interfaces can better adapt to human natural interaction methods, improve interaction efficiency and user experience. Although significant progress has been made in multimodal natural human-machine interfaces, there are still some challenges and future development directions.…”

Section: Theoretical Basismentioning

confidence: 99%

3D Landscape Entity Modeling Based on Geometric Elements and Computer Aided Design

Liu,

Xu,

Jiang

2023

CADandA

View full text Add to dashboard Cite

Traditional landscape design methods are often limited to 2D plane drawings due to technical limitations, and it is difficult to truly and accurately show designers' ideas and expected effects. Computer-aided design (CAD) technology can generate 3D models quickly and accurately by using the powerful computing and processing capabilities of computers, so that designers can view and design landscape space more intuitively. The research goal is to explore a method combining geometric elements, CAD and machine learning (ML) to improve the efficiency of 3D landscape entity modeling. In this article, an optimization algorithm of 3D landscape entity modeling based on ML is proposed. By intelligently adjusting parameters, the algorithm significantly improves the efficiency of the modeling process and ensures the accuracy of the modeling. The results show that the algorithm is excellent in image feature detection and has high accuracy. In addition, the algorithm can also provide suggestions for optimizing the model according to the learning and analysis of the historical model, so that the final landscape model can meet the design requirements more.

show abstract

“…One of the IMUs is located on the back of the hand, and the other five IMUs are respectively located on the second section of each finger. The sensing data of these IMUs can be sent out through wireless communication (Bluetooth) in the data glove [43,44]. Human gestures can be expressed through the attitudes of five fingers and the back of the hand as shown in Figure 3d.…”

Section: Human Handover Intention Sensingmentioning

confidence: 99%

A Novel Human Intention Prediction Approach Based on Fuzzy Rules through Wearable Sensing in Human–Robot Handover

Zou¹,

Liu²,

et al. 2023

Biomimetics

View full text Add to dashboard Cite

With the use of collaborative robots in intelligent manufacturing, human–robot interaction has become more important in human–robot collaborations. Human–robot handover has a huge impact on human–robot interaction. For current research on human–robot handover, special attention is paid to robot path planning and motion control during the handover process; seldom is research focused on human handover intentions. However, enabling robots to predict human handover intentions is important for improving the efficiency of object handover. To enable robots to predict human handover intentions, a novel human handover intention prediction approach was proposed in this study. In the proposed approach, a wearable data glove and fuzzy rules are firstly used to achieve faster and accurate human handover intention sensing (HIS) and human handover intention prediction (HIP). This approach mainly includes human handover intention sensing (HIS) and human handover intention prediction (HIP). For human HIS, we employ wearable data gloves to sense human handover intention information. Compared with vision-based and physical contact-based sensing, wearable data glove-based sensing cannot be affected by visual occlusion and does not pose threats to human safety. For human HIP, we propose a fast handover intention prediction method based on fuzzy rules. Using this method, the robot can efficiently predict human handover intentions based on the sensing data obtained by the data glove. The experimental results demonstrate the advantages and efficacy of the proposed method in human intention prediction during human–robot handover.

show abstract

Multimodal Natural Human–Computer Interfaces for Computer-Aided Design: A Review Paper

Cited by 15 publications

References 140 publications

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

Multimodal Learning‐Based Proactive Human Handover Intention Prediction Using Wearable Data Gloves and Augmented Reality

3D Landscape Entity Modeling Based on Geometric Elements and Computer Aided Design

A Novel Human Intention Prediction Approach Based on Fuzzy Rules through Wearable Sensing in Human–Robot Handover

Contact Info

Product

Resources

About