FlyingHand

Duan, Tinglin; Punpongsanon, Parinya; Iwai, Daisuke; Sato, Kosuke

doi:10.1145/3283254.3283258

Cited by 18 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have been conducted to make people feel as if they are touching objects that they cannot touch with their physical hands, such as remote or distant objects, by combining a substitute hand, such as CG hands or robotic hands, with haptic feedback devices [10], [11]. In the context of ExtendedHand, which is the focus of this study, Tanabe et al [5] and Watanabe et al [6] provided tactile stimuli to the user's hand using a haptic feedback device when the projected extended hand comes into contact with objects, thereby making the user perceive a sense of touch.…”

Section: A Tactile Feedback Of Untouchable Objectsmentioning

confidence: 99%

Responsive-ExtendedHand: Adaptive Visuo-Haptic Feedback Recognizing Object Property With RGB-D Camera for Projected Extended Hand

Sato,

Iwai,

Sato

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

The ExtendedHand interface displays computer graphics (CG) hand images in real space from a projector, allowing the user to visually point at and touch real objects that are out of their physical reach. Furthermore, when the projected CG hand (extended hand) touches an object, the user can feel the tactile sensation of the object through pseudo-haptics by giving the extended hand visual effects that emphasize the action. In the previous psychological study, the human operator had to manually assign the location and shape of objects and the intensities of their visual effects in advance in order to emphasize the appropriate visual effect for the object touched by the extended hand. To increase practical feasibility, we propose an adaptive system that utilizes an RGB-D camera and deep neural networks to generate the appropriate visual effects automatically and apply them to the projected extended hand. By employing U-Net to generate the appropriate intensities of the visual effects from the captured color and depth images, the system can estimate the appropriate visual effects for objects without pre-setting them. The user evaluation results showed that the proposed system allowed users to naturally perceive the tactile sensation of objects at a rate of 44%, instead of the manual rate of 49%. INDEX TERMS Augmented reality, body augmentation, deep neural network, pseudo-haptics.

show abstract

Section: A Tactile Feedback Of Untouchable Objectsmentioning

confidence: 99%

Responsive-ExtendedHand: Adaptive Visuo-Haptic Feedback Recognizing Object Property With RGB-D Camera for Projected Extended Hand

Sato,

Iwai,

Sato

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…For this rich range of domains (e.g., entertainment, companionship, communication, assistance), beyond delivery services [10], drones may play a role as sports coaches [11], tour guides [12,13], and in playing games with people [6,14,15]. They can also support people by helping them navigate [16][17][18], walking them home safely at night [19] or assisting elderly populations [5,20]. In this wide range of use cases, drones interact with users [21] or bystanders [22] alike.…”

Section: Impact Of Drone Design On Human-drone Interactionmentioning

confidence: 99%

Generative Adversarial Networks and Data Clustering for Likable Drone Design

Yamin

Cauchard

2022

Sensors

View full text Add to dashboard Cite

Novel applications for human-drone interaction demand new design approaches, such as social drones that need to be perceived as likable by users. However, given the complexity of the likability perception process, gathering such design information from the interaction context is intricate. This work leverages deep learning-based techniques to generate novel likable drone images. We collected a drone image database (N=360) applicable for design research and assessed the drone’s likability ratings in a user study (N=379). We employed two clustering methodologies: 1. likability-based, which resulted in non-likable, neutral, and likable drone clusters; and 2. feature-based (VGG, PCA), which resulted in drone clusters characterized by visual similarity; both clustered using the K-means algorithm. A characterization process identified three drone features: colorfulness, animal-like representation, and emotional expressions through facial features, which affect drone likability, going beyond prior research. We used the likable drone cluster (N=122) for generating new images using StyleGAN2-ADA and addressed the dataset size limitation using specific configurations and transfer learning. Our results were mitigated due to the dataset size; thus, we illustrate the feasibility of our approach by generating new images using the original database. Our findings demonstrate the effectiveness of Generative Adversarial Networks (GANs) exploitation for drone design, and to the best of our knowledge, this work is the first to suggest GANs for such application.

show abstract

Modifying Texture Perception With Pseudo-Haptic Feedback for a Projected Virtual Hand Interface

et al. 2020

Self Cite

View full text Add to dashboard Cite

Human body augmentation makes it possible to obtain new abilities that we cannot achieve with our actual bodies. A projected virtual hand interface is a promising approach for body augmentation because it can extend a user's reach in daily life without the need to wear a device. Although users can manipulate a projected virtual hand as if it were their own hand and can interact with distant objects through it, they cannot feel the sensation of touch when the projected virtual hand is overlaid on a real object. In this paper, we propose a novel pseudo-haptic feedback framework to provide users with the tactile texture of objects without the use of haptic devices. We designed three types of visual effects that produce unevenness, slipperiness, and softness. The experimental results indicate that the proposed visual effects can lead users to feel the intended tactile sensation. Furthermore, the visual effects provide users with tactile sensations with three to five levels of intensity without producing a strange feeling. INDEX TERMS Body augmentation, projected virtual hand, pseudo-haptic feedback, tactile sensation.

show abstract

FlyingHand

Cited by 18 publications

References 9 publications

Responsive-ExtendedHand: Adaptive Visuo-Haptic Feedback Recognizing Object Property With RGB-D Camera for Projected Extended Hand

Responsive-ExtendedHand: Adaptive Visuo-Haptic Feedback Recognizing Object Property With RGB-D Camera for Projected Extended Hand

Generative Adversarial Networks and Data Clustering for Likable Drone Design

Modifying Texture Perception With Pseudo-Haptic Feedback for a Projected Virtual Hand Interface

Contact Info

Product

Resources

About