Grips and gestures on a multi-touch pen

Abstract

This paper explores the interaction possibilities enabled when the barrel of a digital pen is augmented with a multitouch sensor. We present a novel multi-touch pen (MTPen) prototype and discuss its alternate uses beyond those of a standard stylus, such as allowing new touch gestures to be performed using the index finger or thumb and detecting how users grip the device as a mechanism for mode switching. We also discuss the hardware and software implementation challenges in realizing our prototype, and showcase how one can combine different grips (tripod, relaxed tripod, sketch, wrap) and gestures (swipe and double tap) to enable new interaction techniques with the MTPen in a prototype drawing application. One specific aim is the elimination of some of the comfort problems associated with existing auxiliary controls on digital pens. Mechanical controls such as barrel buttons and barrel scroll wheels work best in only a few specific hand grips and pen rotations. Comparatively, our gestures can be successfully and comfortably performed regardless of the rotation of the pen or how the user grips it, offering greater flexibility in use. We describe a formal evaluation comparing MTPen gestures against the use of a barrel button for mode switching. This study shows that both swipe and double tap gestures are comparable in performance to commonly employed barrel buttons without its disadvantages.

Sensing techniques for tablet+stylus interaction

“…Pens can be augmented with motion [17,48], grip [46,47], and near-surface range sensing [28]. MTPen [46] uses grip sensing to detect a tripod writing grip, or to invoke different brushes.…”

“…MTPen [46] uses grip sensing to detect a tripod writing grip, or to invoke different brushes. Sun et al [47] use an integrated IMU on the pen to assist grip recognition, an approach which we adopt here, and also sense the orientation of the tablet (e.g.…”

“…Our work goes beyond these efforts; compared to the most closely related work (i.e. MTPen [46], FlexAura [28], and [17,21]), the main things that distinguish our contribution are our behavioral observations, the holistic pen and tablet sensing platform, and application of new techniques to well-considered UI challenges.…”

A cuttable multi-touch sensor

“…The commonly used principle for the construction of a multi-touch sensor [17,22,28,20,11,4,18] is illustrated in Fig. 3.…”

3D Navigation with Six Degrees-of-Freedom using a Multi-Touch Display

“…This meant the time the user took between the target collision and the successful sentence completion using the keyboard. Similar approaches to the switch between two devices have been studied [194]. The users' access time is called homing, when working with the Keystroke-Level Model (KLM) [25].…”