Effective 2D Stroke-based Gesture Augmentation for RNNs

Maslych, Mykola; Taranta, Eugene M.; Aldilati, Mostafa; LaViola, Joseph J.

doi:10.1145/3544548.3581358

Cited by 2 publications

(2 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This limitation is particularly problematic because many applications demand custom interaction semantics. Accordingly, the focus has shifted towards recognizing gestures in low-resource or few-shot settings [16,23]. Rahimian et al [23] first explored this few-shot learning setting.…”

Section: Context-aware Gesture Recognitionmentioning

confidence: 99%

“…Rahimian et al [23] first explored this few-shot learning setting. A more recent contribution by Maslych et al [16] significantly enhanced performance by effectively integrating multiple data augmentation strategies. However, although the need for large-scale annotation has been alleviated, these improved methods still necessitate the pre-definition of gesture categories.…”

Section: Context-aware Gesture Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Chinese Agronomy and the Development of Agronomy Concepts

Zeng¹

2021

History of Science and Technology in China

View full text Add to dashboard Cite

Current gesture recognition systems primarily focus on identifying gestures within a predefined set, leaving a gap in connecting these gestures to interactive GUI elements or system functions (e.g., linking a 'thumb-up' gesture to a 'like' button). We introduce GestureGPT, a novel zero-shot gesture understanding and grounding framework leveraging large language models (LLMs). Gesture descriptions are formulated based on hand landmark coordinates from gesture videos and fed into our dual-agent dialogue system. A gesture agent deciphers these descriptions and queries about the interaction context (e.g., interface, history, gaze data), which a context agent organizes and provides. Following iterative exchanges, the gesture agent discerns user intent, grounding it to an interactive function. We validated the gesture description module using public first-view and third-view gesture datasets and tested the whole system in two real-world settings: video streaming and smart home IoT control. The highest zero-shot Top-5 grounding accuracies are 80.11% for video streaming and 90.78% for smart home tasks, showing potential of the new gesture understanding paradigm.CCS Concepts: • Human-centered computing → User interface management systems; • Computing methodologies → Natural language processing.

show abstract

Section: Context-aware Gesture Recognitionmentioning

confidence: 99%

Section: Context-aware Gesture Recognitionmentioning

confidence: 99%

Chinese Agronomy and the Development of Agronomy Concepts

Zeng¹

2021

History of Science and Technology in China

View full text Add to dashboard Cite

show abstract

Transforming Hand Gesture Recognition Into Image Classification Using Data Level Fusion

Yusuf,

Habib,

Moustafa

2023

Global Perspectives on Robotics and Autonomous Systems

View full text Add to dashboard Cite

Hand gesture recognition (HGR) is a form of perceptual computing with applications in human-machine interaction, virtual/augmented reality, and human behavior analysis. Within the HGR domain, several frameworks have been developed with different combinations of input modalities and neural network architectures to varying levels of efficacy. Such frameworks maximized performance at the expense of increased computational and hardware requirements. These drawbacks can be mitigated by a skeleton-based framework that transforms the hand gesture recognition task into an image classification task. This chapter explores several temporal information condensation (via data-level fusion) methods for encoding dynamic gesture information into static images. The efficacies of these methods are compared, and the best ones are aggregated into a generalized HGR framework which was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 benchmark datasets. The framework's performance shows competitiveness compared to other frameworks within the state-of-the-art for the datasets.

show abstract

Effective 2D Stroke-based Gesture Augmentation for RNNs

Cited by 2 publications

References 69 publications

Chinese Agronomy and the Development of Agronomy Concepts

Chinese Agronomy and the Development of Agronomy Concepts

Transforming Hand Gesture Recognition Into Image Classification Using Data Level Fusion

Contact Info

Product

Resources

About