A enhanced interaction framework based on VR, AR and MR in digital twin

Ke, Shiqian; Feng, Xiang; Zhi, Zhang; Zuo, Ying

doi:10.1016/j.procir.2019.04.103

Cited by 63 publications

(31 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, it is necessary to provide users with an easy and intuitive interface to display the data included in the DT. The most studied technologies in the I4.0 to solve this problem are the eXtended Reality (XR) devices, such as Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality [61,72,75,76]. In recent years, many researchers have proposed a conceptual or practical use of these technologies to improve the use of DT.…”

Section: Dt Improvements Brought About By Extended Reality Technologiesmentioning

confidence: 99%

“…There are two main types of XR supports that can be found in the literature (see Table 6). The first type is the handheld device (HHD) (see Figure 6a), such as a smartphone or a tablet [22,47,53,55,68,69,75,77,81], which is often already used in industry. Even if the HDD can be used to display augmentations, their main disadvantage is that they need to be supported, either by the user's hand or with a secure support.…”

Section: Main Devices Used For Visualisationmentioning

confidence: 99%

“…This type is mainly used to provide an immersive environment for its user, leaving his or her hands free to interact with the real environment. There are two main subtypes of HMD that can be found in the literature: the optical and/or video see-through HMDs, which let the user see the real-word with immersive augmentations [25,26,43,47,66,67,[70][71][72][73]75,78,80] (see Figure 6b), and the occluded HMDs, where the user is entirely immersed in a virtual environment [25,47,75,78,79] (see Figure 6c). While the occluded HMDs are mainly seen as VR devices, the see-through HMDs are mostly used for AR and MR applications.…”

Section: Main Devices Used For Visualisationmentioning

confidence: 99%

“…However, in an industrial environment, external factors can disrupt these interactions (e.g., dust that disturbs image recognition and noise that complicates speech recognition). Other interaction methods can be used to overcome this problem, such as using the location of the user to display specific augmentations [25,47,75,76].…”

Section: Interaction With the Modelmentioning

confidence: 99%

See 3 more Smart Citations

BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review

Noblecourt²,

et al. 2021

View full text Add to dashboard Cite

In recent years, the use of digital twins (DT) to improve maintenance procedures has increased in various industrial sectors (e.g., manufacturing, energy industry, aerospace) but is more limited in the construction industry. However, the operation and maintenance (O&M) phase of a building’s life cycle is the most expensive. Smart buildings already use BIM (Building Information Modeling) for facility management, but they lack the predictive capabilities of DT. On the other hand, the use of extended reality (XR) technologies to improve maintenance operations has been a major topic of academic research in recent years, both through data display and remote collaboration. In this context, this paper focuses on reviewing projects using a combination of these technologies to improve maintenance operations in smart buildings. This review uses a combination of at least three of the terms “Digital Twin”, “Maintenance”, “BIM” and “Extended Reality”. Results show how a BIM can be used to create a DT and how this DT use combined with XR technologies can improve maintenance operations in a smart building. This paper also highlights the challenges for the correct implementation of a BIM-based DT combined with XR devices. An example of use is also proposed using a diagram of the possible interactions between the user, the DT and the application framework during maintenance operations.

show abstract

Section: Dt Improvements Brought About By Extended Reality Technologiesmentioning

confidence: 99%

Section: Main Devices Used For Visualisationmentioning

confidence: 99%

Section: Main Devices Used For Visualisationmentioning

confidence: 99%

Section: Interaction With the Modelmentioning

confidence: 99%

See 2 more Smart Citations

BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review

Noblecourt²,

et al. 2021

View full text Add to dashboard Cite

show abstract

“…There is currently a multitude of methods and techniques to help designers to graphically represent a product. Among the most commonly used are rendering and animation techniques, although it should be noted that advanced techniques have recently appeared such as VR and AR [30,31]. In addition, other methods are being developed that focus on the choice of sounds, such as the door of a car when it closes [32] or the sound of an office chair as it slides [33].…”

Section: Introductionmentioning

confidence: 99%

Haptic Hybrid Prototyping (HHP): An AR Application for Texture Evaluation with Semantic Content in Product Design

et al. 2019

View full text Add to dashboard Cite

The manufacture of prototypes is costly in economic and temporal terms and in order to carry this out it is necessary to accept certain deviations with respect to the final finishes. This article proposes haptic hybrid prototyping, a haptic-visual product prototyping method created to help product design teams evaluate and select semantic information conveyed between product and user through texturing and ribs of a product in early stages of conceptualization. For the evaluation of this tool, an experiment was realized in which the haptic experience was compared during the interaction with final products and through the HHP. As a result, it was observed that the answers of the interviewees coincided in both situations in 81% of the cases. It was concluded that the HHP enables us to know the semantic information transmitted through haptic-visual means between product and user as well as being able to quantify the clarity with which this information is transmitted. Therefore, this new tool makes it possible to reduce the manufacturing lead time of prototypes as well as the conceptualization phase of the product, providing information on the future success of the product in the market and its economic return.

show abstract

A Hybrid Upper‐Arm‐Geared Exoskeleton with Anatomical Digital Twin for Tangible Metaverse Feedback and Communication

Ha,

Lee,

Cho

et al. 2023

Adv Materials Technologies

View full text Add to dashboard Cite

The pandemic coincided with rapid advancements in virtual reality (VR) and mixed reality (MR) in healthcare. The idea of virtually replicating the real world and its associated experiences has garnered significant attention under the newly coined term “metaverse.” The metaverse serves as a communication platform that integrates physical and virtual experiences. However, the lack of physical interaction between users and virtual environments remains an obstacle. This study introduces a hybrid upper‐arm‐geared exoskeleton system that combines an anatomical digital‐twin model with tangible VR torque feedback and an MR remote healthcare monitoring in a musculoskeletal interface device format. The device employs a dual epicyclic geared motor actuator capable of exerting torque feedback. It derived torque profiles that simulate physical interactions with hysteretic damping in the virtual environment. An anatomical digital twin model is also incorporated into the fabricated device in an MR format and assessed the device's performance using electromyogram and thermographic sensors. In addition, it evaluated haptic fidelity and versatility as an immersive metaverse wearable and performed support vector machine‐based analysis for motion feature classification. It believes this approach has the potential to be highly beneficial for providing tangible metaverse feedback and communication combined with a human‐informed digital‐twin model for remote healthcare monitoring in the postpandemic era.

show abstract

A enhanced interaction framework based on VR, AR and MR in digital twin

Cited by 63 publications

References 13 publications

BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review

BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review

Haptic Hybrid Prototyping (HHP): An AR Application for Texture Evaluation with Semantic Content in Product Design

A Hybrid Upper‐Arm‐Geared Exoskeleton with Anatomical Digital Twin for Tangible Metaverse Feedback and Communication

Contact Info

Product

Resources

About