Simulating Realistic Human Motion Trajectories of Mid-Air Gesture Typing

Shen, Junxiao; Dudley, John J.; Kristensson, Per Ola

doi:10.1109/ismar52148.2021.00056

Cited by 15 publications

(6 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the controllers' internal sensors enable higher tolerance to occlusions and faster response to motion. Professional systems such as OptiTrack Motion Capture provide better accuracy and faster motion detection for hand tracking [20,30,81], but they are mostly not available in mobile XR workspaces with consumer XR hardware. Kern et al [40] showed that most XR controllers can also be used as XR styluses.…”

Section: Xr Controllers and Hand Trackingmentioning

confidence: 99%

“…Since our contact-based input techniques enable higher or at least similar text input performance after only a few typed sentences, compared to distance-based approaches [8,84], the question arises of whether distance-based techniques have reached an upper limit, while contact-based approaches can surpass it. One can imagine how fast text input might be with accurate 10-finger tracking [20,30,81].…”

Section: Task Performancementioning

confidence: 99%

See 1 more Smart Citation

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality

Kern¹,

Niebling

Latoschik³

2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1: The virtual keyboard layout is based on the traditional layout of physical keyboards. (A) Shows the tap keyboard in video see-through augmented reality (VST AR) and (B) the swipe keyboard in virtual reality (VR) with a black tail following the user's controller. For the VR condition, we created an accurate digital twin of the real physical room. In the VST AR condition, the real world was captured by the video see-through capabilities of the XR device. To enable precise interaction, the user holds the controller upside down in a pen-like posture and presses/swipes the virtual keyboard keys with the bottom tip of the controller. Three word suggestions are displayed in the upper left. The arrow keys to move the cursor and the backspace key are located in the upper right. The cursor can also be positioned directly in the input field. The shift keys and the space bar are hidden on the swipe keyboard because they are not required for text entry. Rectangular 3D keys provide visual feedback, and when pressed, the controllers vibrate.

show abstract

Section: Xr Controllers and Hand Trackingmentioning

confidence: 99%

Section: Task Performancementioning

confidence: 99%

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality

Kern¹,

Niebling

Latoschik³

2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Inter‐joint‐variation relates to the correctness of skeletal motion by evaluating if a generated motion has the same inter‐joint relationships as captured motion, while motion energy was defined as the average amount of motion in a sequence leveraging the fact that motion sequences with faster dynamics will have more energy than the ones with slower dynamics. Shen et al [SDK21] presented another local‐body motion level approach to evaluate the realism of hand trajectories. They considered geometric micro metrics (e.g., curvature, aspect, vicinity curliness, linearity, and slope) to evaluate the low‐level features of the trajectories.…”

Section: Evaluation Methodsmentioning

confidence: 99%

“…Authors proposed to evaluate their resulting animation in comparison to ground‐truth data both using a distance metrics based on DTW but also comparing jerk results as a good indicator of motion naturalness. Finally, at the kinematics level, we can mention the study of Shen et al [SDK21] focusing on hand motions. In order to have accurate human‐like hand trajectories, authors proposed to define a minimum trajectory error to be tolerated, allowing the generated trace to be recognized as human one.…”

Section: Evaluation Methodsmentioning

confidence: 99%

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

Rekik,

Wuhrer,

Hoyet

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

Generating realistic animated virtual humans is a problem that has been extensively studied with many applications in different types of virtual environments. However, the creation process of such realistic animations is challenging, especially because of the number and variety of influencing factors, that should then be identified and evaluated. In this paper, we attempt to provide a clearer understanding of how the multiple factors that have been studied in the literature impact the level of realism of animated virtual humans, by providing a survey of studies assessing their realism. This includes a review of features that have been manipulated to increase the realism of virtual humans, as well as evaluation approaches that have been developed. As the challenges of evaluating animated virtual humans in a way that agrees with human perception are still active research problems, this survey further identifies important open problems and directions for future research.

show abstract

“…to generate realistic samples in image [12,43,50], speech [19], and motion trajectory [41] synthesis. We adopted the Imaginative GAN from Shen et al [40] as one of our data augmentation strategies, which can approximate the true distribution of the input data and sample new data from the approximated distribution.…”

Section: Augmentationmentioning

confidence: 99%

Gesture Spotter: A Rapid Prototyping Tool for Key Gesture Spotting in Virtual and Augmented Reality Applications

Shen

Dudley

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

In this paper we examine the task of key gesture spotting: accurate and timely online recognition of hand gestures. We specifically seek to address two key challenges faced by developers when integrating key gesture spotting functionality into their applications. These are: i) achieving high accuracy and zero or negative activation lag with single-time activation; and ii) avoiding the requirement for deep domain expertise in machine learning. We address the first challenge by proposing a key gesture spotting architecture consisting of a novel gesture classifier model and a novel single-time activation algorithm. This key gesture spotting architecture was evaluated on four separate hand skeleton gesture datasets, and achieved high recognition accuracy with early detection. We address the second challenge by encapsulating different data processing and augmentation strategies, as well as the proposed key gesture spotting architecture, into a graphical user interface and an application programming interface. Two user studies demonstrate that developers are able to efficiently construct custom recognizers using both the graphical user interface and the application programming interface.

show abstract

Simulating Realistic Human Motion Trajectories of Mid-Air Gesture Typing

Cited by 15 publications

References 35 publications

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

Gesture Spotter: A Rapid Prototyping Tool for Key Gesture Spotting in Virtual and Augmented Reality Applications

Contact Info

Product

Resources

About