Designing Hand Pose Aware Virtual Keyboard With Hand Drift Tolerance

Abstract

An unintentional hand drift adversely affects the typing performance of conventional virtual keyboards. To overcome this, we proposed to utilize the typing patterns of skilled typists. First, as most typists enter the keys in the same column with a predetermined finger only, we restricted these keys to be typed by their corresponding fingers. Second, our investigation of skilled typists demonstrated that hand poses vary when the typists touch different keys. Thus, rather than locating the touch point as in the case of existing virtual keyboards, we attempted to use unique hand poses to infer the target key. Based on these two techniques, we implemented a novel hand poses aware virtual keyboard that is tolerant of hand drift. Our experimental studies yielded the following results: 1) most of the QWERTY-familiar typists who have varying typing habits were easily adaptable to the proposed keyboard design and 2) the proposed keyboard outperformed existing virtual keyboards in terms of typing speed and several error rates, and eventually achieved a typing speed of approximately 56 WPM.INDEX TERMS Eyes-free, hand drift, hands, text entry, touch typing, virtual keyboard, virtual reality.