Quadmetric Optimized Thumb-to-Finger Interaction for Force Assisted One-Handed Text Entry on Mobile Headsets

Abstract

Augmented reality head-worn computers often feature small-sized touch interfaces that complicate interaction with content, provide insufficient space for comfortable text input, and can be awkward to use in social situations. This paper presents a novel one-handed thumb-to-finger text entry solution for augmented reality head-worn computers. We design a glove composed of 12 force-sensitive nodes featuring an ambiguous keyboard layout. We first explore the viability of force disambiguation to evaluate the force division within the force spectrum. We select a 3-level force division as it allows to considerably reduce the number of keys while featuring a high (83.9%) accuracy. Following this pilot study, we map the 26 English characters onto the 9 nodes located on the index, middle and ring fingers in a 3-3-3 configuration, and attribute the space, enter and backspace keys to the remaining three nodes. We consider text entry performance as a quadmetric optimization problem considering the following criteria: goodness of character pairs, layout similarity to the QWERTY keyboard, easiness of force interaction, and comfort level of thumb reach. The resulting layout strikes a balance between performance and usability. We finally evaluate the quadmetric optimized layout over 6 sessions with 12 participants. The participants achieve an average text entry rate of 6.47 WPM with 6.85% error rate in the final session, which is significantly faster than existing thumb-to-finger solutions. In addition, our one-handed text entry system enhances the user mobility compared to other state-of-the-art solutions by freeing one hand, while allowing the user to direct his visual attention to other activities.

M[eye]cro

“…To recognize thumb-to-finger gestures several approaches use cameras, however, they require calibration and they suffer from occlusion (i.e occlusion of joints and finger segments) [29,36,41]. Other approaches equip the hand with touch sensors, which seem to be a more reliable and easy to design approach [21,28,45,52,56,57]. This is the chosen method for our technique M[eye]cro.…”

“…However, this technique was not quantitatively compared to another technique. The other techniques were designed for text entry using thumb-to-finger gestures [21,28,49,52,55]. All these techniques were compared to standard keyboards, but most importantly none were tested in a multitasking context.…”

Emerging ExG-based NUI Inputs in Extended Realities: A Bottom-up Survey

“…Similarly, the location of touch-based inputs can proceed with some noticeable approaches, such as the spectacle frame of immersive reality headsets [220], handheld controller [223], as well as the surroundings of the headsets [129,133]. Recent works demonstrate more concealed operations for immersive reality in mobile environments, with on-body interaction [131], including but not limited to finger-to-forearm interaction [60], finger space interaction [132], and thumb-to-index finger interaction [218].…”

“…Third, the fact that human thumbs can reach other fingers is employed to formulate a keyboard within a palm area [132]. An additional example is that the input of muscle tension drives the manipulation of three-dimesional (3D) objects and the corresponding pseudo-haptic (visual) feedback [187].…”

MyoKey: Surface Electromyography and Inertial Motion Sensing-based Text Entry in AR

“…On-device interaction [8] enables character selection, for example, on a sensible surface of a device: Taps and Swipes [9] on the spectacle frame of smartglasses, and an addendum ring surface allowing cursor pointing to characters within finger space [10] for subtle inputs [11]. The body-center interaction method is based on an interface attached to the user's body.…”