GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications

Wang, Tianyi; Qian, Xun; He, Fengming; Hu, Xiyun; Cao, Yichao; Ramani, Karthik

doi:10.1145/3472749.3474769

Cited by 36 publications

(16 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by prior research, we believe Extended Reality (XR) could provide an effective medium to communicate test cases for validation to users. Enabled by spatial awareness of XR, prior works have used virtual augmentations adjacent to relevant physical entities to facilitate end-users to understand complicated contexts such as human activities [13,92], interactions [12,40,53,91], and status of smart objects [41]. Further, while being immersed in XR environments, end-users can experience simulated scenarios free from the limitations of physical contexts such as user status and time.…”

Section: Immersiveness In Xrmentioning

confidence: 99%

Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended Reality

Qian,

Wang,

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Meta Reality Labs Research (a) (b) (c) (d) Figure 1: Overview of the Fast-Forward Reality workflow: (a) An end-user initiates an authoring session of a context-aware policy (CAP) while being immersed in an Extended Reality (XR) authoring environment. The user includes context instances into the CAP by selecting the corresponding XR icons. (b) The system adaptively generates multiple unit test cases based on the context history of the user's everyday routines and the CAP, and highlights them with XR visualizations. (c) The user acts out the suggested test cases by in-situ actions and selections of the involved context instances to intuitively validate whether the CAP reacts as expected under each test case. (d) The user refines the CAP after noticing potential improvements and repeats the validation until the CAP is error-free under all test cases.

show abstract

Section: Immersiveness In Xrmentioning

confidence: 99%

Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended Reality

Qian,

Wang,

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…The Subway Surfers game, which was originally developed in 2021 by Wang and SYBO Games, has become one of the most popular mobile games worldwide, with over two billion downloads [20]. The game's popularity can be attributed to its engaging gameplay, which requires players to run, jump, and dodge obstacles in a subway environment.…”

Section: Related Workmentioning

confidence: 99%

Applying Artificial Intelligence and Computer Vision for Augmented Reality Game Development in Sports

Omarov,

Baibaktina

et al. 2023

IJACSA

View full text Add to dashboard Cite

This paper delineates the intricate process of crafting an Augmented Reality (AR)-enriched version of the Subway Surfers game, engineered with an emphasis on action recognition and the leverage of Artificial Intelligence (AI) principles, with the primary objective of boosting children's enthusiasm towards physical activity. The gameplay, fundamentally predicated on advanced computer vision methodologies for discerning player kinesthetics, and reinforced with machine learning tactics for modulating the intricacy of the game in accordance with player capabilities, offers an immersive and engaging interface. This innovative amalgamation serves to not only catalyze children's interest in participating in active exercises, but also introduces a playful aspect to it. The procedural development of the game required the cohesive assimilation of a diverse spectrum of technologies, encompassing Unity for game development, TensorFlow for implementing machine learning algorithms, and Vuforia for crafting the AR elements. A preliminary study, conducted to assess the efficacy of the game in fostering a pro-sport attitude in children, reported encouraging outcomes. Given the potential of the game to incite physical activity among young users, it could be construed as a promising antidote to sedentarism and a potent catalyst for endorsing a healthier lifestyle.

show abstract

“…Wang et al created the GesturAR on Hololens2 using Unity3D. Users can complete freehand interaction by using the visual programming interface to match gestures with the response of AR virtual content [11]. Unlike other works, users design gestures to interact with virtual elements completely on their own in GesturAR, which improves the flexibility and independence of freehand interaction (Figure 3).…”

Section: Embodied Ar Presentationmentioning

confidence: 99%

“…For example, Wang et al have created a gesture AR authoring system, in which users completes freehand interaction using a visual programming interface to match gestures with the response of AR virtual content. It improves the flexibility and independence of freehand interaction [11]. However, more researchers are inclined to study user-defined gestures with the continuous development and maturity of gesture recognition technology.…”

Section: Introductionmentioning

confidence: 99%

Classification, Application, Challenge, and Future of Midair Gestures in Augmented Reality

Gong

Zhou

et al. 2022

Journal of Sensors

View full text Add to dashboard Cite

Augmented Reality (AR) technology provides many opportunities to enhance people’s experience in interacting with data. Midair gesture, a natural interaction mode in AR, interacts with virtual elements without any auxiliary devices. It has become a hot topic of interest for researchers with the development of gesture recognition technology. From the perspective of user experience, the types of midair gesture, gesture recognition technology, applications were discussed. The challenges of air gesture interaction from two aspects of user experience and technology were analyzed, and the importance of collaborative gesture and low-cost user experience in the future were emphasized. Finally, the application prospect of air gesture interaction distance education, medical health, industry, office, and so on is discussed.

show abstract

GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications

Cited by 36 publications

References 76 publications

Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended Reality

Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended Reality

Applying Artificial Intelligence and Computer Vision for Augmented Reality Game Development in Sports

Classification, Application, Challenge, and Future of Midair Gestures in Augmented Reality

Contact Info

Product

Resources

About