A cross-device drag-and-drop technique

Abstract

Many interactions naturally extend across smart-phones and devices with larger screens. Indeed, data might be received on the mobile but more conveniently processed with an application on a larger device, or vice versa. Such interactions require spontaneous data transfer from a source location on one screen to a target location on the other device. We introduce a cross-device Drag-andDrop technique to facilitate these interactions involving multiple touchscreen devices, with minimal effort for the user. The technique is a two-handed gesture, where one hand is used to suitably align the mobile phone with the larger screen, while the other is used to select and drag an object between devices and choose which application should receive the data.

Cross-Device Taxonomy

“…Many of the classic cross-device interaction techniques -inspired by the seminal Hyperdrag [281] and Pick-and-drop [223,278] -use direct touch or mouse interaction with the displays to move content from one device to another. Examples include using drag-and-drop across the bezel of multiple screens [312], swiping in the direction of another device [159,348], as well as panning [270], tapping [310], and flicking gestures [277]. Direct interaction with content across devices has been supported through drag-and-drop proxy icon portals [64,123,206], pressurebased press-and-flick techniques [223], pinch-to-zoom across multiple displays [206], or drag-and-pop and drag-and-pick techniques for multi-screen environments [15].…”

Technical framework supporting a cross-device drag-and-drop technique

“…The application itself is composed of two components: one running on the mobile device and the other running as a desktop application. To demonstrate how an application could be designed to accommodate cross-device Drag-and-Drop gestures, we have also designed a custom mobile email client that can respond to events originating from a different device [7]. Both applications are built on the same framework and are targeted for use on configurations involving a desktop system and a mobile device both belonging to the same user.…”

“…It is also used to allow users to drag items from the desktop system to the mobile. These two applications and user feedback of the deployed systems are described in [7]. In the following sections, the technical implementation of the framework is described.…”

Select & Apply: understanding how users act upon objects across devices

“…10). A similar technique allowing touch Drag-and-Drop gestures between desktop PCs and mobile devices has been presented by Simeone et al [29]. Cross-Device Touch Drag-and-Drop: (left) the user holds the source device next to one of the edges of the target device; s/he then proceeds to select and drag an item towards the edges of the screen and into the target device's screen; upon reaching the intended location on the target device's screen, the user releases her/his finger to finalize the action (right).…”

“…in a home/work network, the time between exiting the source device's screen and entering the target device's screen would be sufficient for most circumstances such as transferring text paragraphs, phone numbers, contacts, etc. In [29] the technique is implemented by means of either applications that are aware of the presence of other devices or through an intermediary one that allows cross-device communication within sandboxed operating environments.…”