Press-n-Paste: Copy-and-Paste Operations with Pressure-sensitive Caret Navigation for Miniaturized Surface in Mobile Augmented Reality

Abstract

Copy-and-paste operations are the most popular features on computing devices such as desktop computers, smartphones and tablets. However, the copy-and-paste operations are not sufficiently addressed on the Augmented Reality (AR) smartglasses designated for real-time interaction with texts in physical environments. This paper proposes two system solutions, namely Granularity Scrolling (GS) and Two Ends (TE), for the copy-and-paste operations on AR smartglasses. By leveraging a thumb-size button on a touch-sensitive and pressure-sensitive surface, both the multi-step solutions can capture the target texts through indirect manipulation and subsequently enables the copy-and-paste operations. Based on the system solutions, we implemented an experimental prototype named Press-n-Paste (PnP). After the eight-session evaluation capturing 1,296 copy-and-paste operations, 18 participants with GS and TE achieve the peak performance of 17,574 ms and 13,951 ms per copy-and-paste operation, with 93.21% and 98.15% accuracy rates respectively, which are as good as the commercial solutions using direct manipulation on touchscreen devices. The user footprints also show that PnP has a distinctive feature of miniaturized interaction area within 12.65 mm * 14.48 mm. PnP not only proves the feasibility of copy-and-paste operations with the flexibility of various granularities on AR smartglasses, but also gives significant implications to the design space of pressure widgets as well as the input design on smart wearables.

An Exploration of Hands-free Text Selection for Virtual Reality Head-Mounted Displays

“…In this work, we focused on comparing the dwell-based selection mechanism by using eye blinks and voice. Both of them are relatively easy to do, give more control to users, and, more importantly, can be captured by a wide range of HMDs without additional hardware requirements or external sensing (handheld) devices, unlike some recent work [6,26,27], which can add extra costs to users and complexity to the interaction process.…”

Theophany

“…A more realistic approach would consist of integrating all the components within smaller-scale smartglasses, with the added advantage of overlaying subtitles on top of the participants' physical location. However, with the state of current technologies [15], we consider that using a combination of Microsoft Hololens 2 and smartphones represents a good emulation of the setting for demonstration purposes [16].…”