PointerPhone: Using Mobile Phones for Direct Pointing Interactions with Remote Displays

Abstract

Abstract.Large screens or projections in public and private settings have become part of our daily lives, as they enable the collaboration and presentation of information in many diverse ways. When discussing the shown information with other persons, we often point to a displayed object with our index finger or a laser pointer in order to talk about it. Although mobile phone-based interactions with remote screens have been investigated intensively in the last decade, none of them considered such direct pointing interactions for application in everyday tasks. In this paper, we present the concept and design space of PointerPhone which enables users to directly point at objects on a remote screen with their mobile phone and interact with them in a natural and seamless way. We detail the design space and distinguish three categories of interactions including low-level interactions using the mobile phone as a precise and fast pointing device, as well as an input and output device. We detail the category of widgetlevel interactions. Further, we demonstrate versatile high-level interaction techniques and show their application in a collaborative presentation scenario. Based on the results of a qualitative study, we provide design implications for application designs.

SleeD

“…They proposed a collection of interaction techniques in areas such as data transfer, personalization, or localized feedback. This research is expanded upon by Seifert et al [28], who investigate UI mechanisms for moving data from a mobile phone to a table. One of the issues they identified is that "only one hand is available for interacting with the table" while holding the phone.…”

Making VR work

“…Controlling virtual widgets with a tracked, unseen touch screen was explored by Steed et al [21]. Seifert et al [19] extensively investigated using smartphones as pointers to large displays. Song et al [20] defined the concept of a handle bar metaphor to manipulate virtual objects at a distance.…”

Pointing at a Distance with Everyday Smart Devices

“…Early work in this domain leveraged the touchscreen for input, essentially using the device as a drawing pad [23], a mouse plus sketch surface [39], or a touchpad [2,16]. More recently, the ubiquity of accelerometers and gyroscopes in these devices has made it possible to mimic freehand pointing techniques [18,26,33].…”