Digits

Abstract

Figure 1: A) Digits is a wrist-worn sensor for freehand 3D interactions on the move. By instrumenting only the wrist, the user's entire hand is left to interact freely without wearing a data glove. B and C) Digits recovers the full 3D pose of a user's hand. D) Spatial interactions using a mobile phone and Digits. ABSTRACTDigits is a wrist-worn sensor that recovers the full 3D pose of the user's hand. This enables a variety of freehand interactions on the move. The system targets mobile settings, and is specifically designed to be low-power and easily reproducible using only off-the-shelf hardware. The electronics are self-contained on the user's wrist, but optically image the entirety of the user's hand. This data is processed using a new pipeline that robustly samples key parts of the hand, such as the tips and lower regions of each finger. These sparse samples are fed into new kinematic models that leverage the biomechanical constraints of the hand to recover the 3D pose of the user's hand. The proposed system works without the need for full instrumentation of the hand (for example using data gloves), additional sensors in the environment, or depth cameras which are currently prohibitive for mobile scenarios due to power and form-factor considerations. We demonstrate the utility of Digits for a variety of application scenarios, including 3D spatial interaction with mobile devices, eyes-free interaction on-the-move, and gaming. We conclude with a quantitative and qualitative evaluation of our system, and discussion of strengths, limitations and future work.

Expressy

“…However, they require that the user wears potentially cumbersome and uncomfortable gloves. Alternative technologies that could allow qualities of hand and finger movement to be tracked to enrich touch input without the need to wear a glove include arm-mounted piezoelectric sensors [14] and wrist-mounted cameras [27]. Benko et al [3] used EMG to sense users' hand movements and demonstrate a range of enhanced touch interactions, which included pressure detection, contact finger identification and off-surface pinch, throw and flick gestures.…”

“…While some of the technologies discussed are aimed at making performance of common discrete commands quicker or more natural [5,6,11,13,14,19,21,24,29], others demonstrate a potential to support more expressive interactions [22,35]. However, the majority of these approaches rely on bespoke or impractical hardware configurations [3,10,27,32,44] and many only extend the expressiveness of touch interaction with a single or small number of new capabilities [12,17,20,26,38,42,46,47]. Moreover, none of them introduce a conceptual model for expressive touch interactions that can guide other researchers interested in this field of research and stimulate new ideas for how such technologies can be used.…”

Around-device devices

“…This could be a worn color glove [40] or markers put on objects. Instead of a glove, Digits has users wear a device on the wrist that is able to track hand gestures [25]. Devices incorporating magnetometers, can track fingers wearing magnetic rings [15] or users holding magnets or objects containing magnets [24].…”

An Easily Customized Gesture Recognizer for Assisted Living Using Commodity Mobile Devices

“…The visual recording devices are usually installed at a fixed location and the gesture recognition is restricted in confined space. Wearable devices used for visual recognition include glasses camera [13] and wrist-worn device with infrared spectral camera (IR) [14]. …”