Efficient 6-DoF Tracking of Handheld Objects from an Egocentric Viewpoint

Pandey, Rohit; Pidlypenskyi, Pavel; Yang, Shao-Wen; Kaeser-Chen, Christine

doi:10.1007/978-3-030-01216-8_26

Cited by 6 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work on egocentric computer vision has concerned object recognition [6,26,28], activity recognition [3,10,29,31,34,37,39,50], gaze prediction [21,27,43,51], video summarization [19,35,40], head motion signatures [36], and human pose estimation [23,49]. In particular, many papers have considered problems related to hands in egocentric video, including hand pose estimation [16,30], hand segmentation [4,46], handheld controller tracking [33], and hand gesture recognition [8]. However, to our knowledge, no prior work has considered egocentric hand-based person identification.…”

Section: Related Workmentioning

confidence: 99%

Whose hand is this? Person Identification from Egocentric Hand Gestures

Tsutsui

Crandall

2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Recognizing people by faces and other biometrics has been extensively studied in computer vision. But these techniques do not work for identifying the wearer of an egocentric (first-person) camera because that person rarely (if ever) appears in their own first-person view. But while one's own face is not frequently visible, their hands are: in fact, hands are among the most common objects in one's own field of view. It is thus natural to ask whether the appearance and motion patterns of people's hands are distinctive enough to recognize them. In this paper, we systematically study the possibility of Egocentric Hand Identification (EHI) with unconstrained egocentric hand gestures. We explore several different visual cues, including color, shape, skin texture, and depth maps to identify users' hands. Extensive ablation experiments are conducted to analyze the properties of hands that are most distinctive. Finally, we show that EHI can improve generalization of other tasks, such as gesture recognition, by training adversarially to encourage these models to ignore differences between users.

show abstract

Section: Related Workmentioning

confidence: 99%

Whose hand is this? Person Identification from Egocentric Hand Gestures

Tsutsui

Crandall

2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…Hand pose estimation has drawn increasing attention for decades [1,3,9,5,37,32,6,21,25,36,40]. Re-cent research efforts can be categorized by their input data forms, which primarily include 2D RGB images and 3D RGBD images with depth map information.…”

Section: Vision-based Hand Pose Estimationmentioning

confidence: 99%

Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation

Chen¹,

Lin²,

Xie³

et al. 2018

Preprint

View full text Add to dashboard Cite

Hand pose estimation from a monocular RGB image is an important but challenging task. A main factor affecting its performance is the lack of a sufficiently large training dataset with accurate hand-keypoint annotations. In this work, we circumvent this problem by proposing an effective method for generating realistic hand poses, and show that state-of-the-art algorithms for hand pose estimation can be greatly improved by utilizing the generated hand poses as training data. Specifically, we first adopt an augmented reality (AR) simulator to synthesize hand poses with accurate hand-keypoint labels. Although the synthetic hand poses come with precise joint labels, eliminating the need of manual annotations, they look unnatural and are not the ideal training data. To produce more realistic hand poses, we propose to blend a synthetic hand pose with a real background, such as arms and sleeves. To this end, we develop tonality-alignment generative adversarial networks (TAGANs), which align the tonality and color distributions between synthetic hand poses and real backgrounds, and can generate high quality hand poses. We evaluate TAGAN on three benchmarks, including the RHP, STB, and CMU-PS hand pose datasets. With the aid of the synthesized poses, our method performs favorably against the state-ofthe-arts in both 2D and 3D hand pose estimations.

show abstract