Smartcasting

Abstract

We introduce and formally evaluate smartcasting: a smartphone-based Ray Casting implementation for 3D environments presented on large, public, autostereoscopic displays. By utilizing a smartphone as an input device, smartcasting enables "walk up and use" interaction with large displays, without the need for expensive tracking systems or specialized pointing devices. Through an empirical validation we show that the performance and precision of smartcasting is comparable to a Wiimotebased raycasting implementation, without requiring specialized hardware or high-precision cameras to enable user interaction.

HoloBar: Rapid Command Execution for Head-Worn AR Exploiting Around the Field-of-View Interaction

“…One of the first works adopting this approach focused on the use of multiple devices to improve the interaction with whiteboards [49]. A large part of the works related to this area aims to improve and facilitate interaction with large displays: content sharing [15,42,35], pointing [29], 3D manipulations [47,45,46]. In the XR field, researchers identified quite early the potential of combining mobile devices with HMDs: Feiner et al [21] in 1997 used a tablet to display a second-level menu as well as detailed information about augmented data in the HMD to compensate for its low resolution.…”

Pointing at a Distance with Everyday Smart Devices

“…Early work in this domain leveraged the touchscreen for input, essentially using the device as a drawing pad [23], a mouse plus sketch surface [39], or a touchpad [2,16]. More recently, the ubiquity of accelerometers and gyroscopes in these devices has made it possible to mimic freehand pointing techniques [18,26,33].…”

“…Recent research has shown that freehand pointing with a phone performs on par with the Nintendo Wiimote [26], and that freehand pointing with a smartwatch performs at near parity with Vicon-based freehand distal pointing [18]. As researchers, we believe that distal pointing with ubiquitous personal devices holds the most near-term promise for everyday public display interaction: Essentially, rather than augmenting the entire world or requiring potential users to identify and obtain relevant objects, if the input device is either a smartphone or a smartwatch, the vast majority of people already own and have with them their interaction platform.…”

“…Alongside this, however, there are concerns associated with some techniques: for example, Katsuragawa et al's Watchpoint system set-up has participants wear the smartwatch on their preferred hand [18], presumably at odds with common practice. Furthermore, smartphonebased techniques that use the smartphone as a pointing device also assume a device held by the preferred hand [26], whereas the use of a smartphone as a touchpad proxy would assume the user holds the device with the non-preferred hand [2].…”

Exploring At-Your-Side Gestural Interaction for Ubiquitous Environments

“…Free-space gestural input is an active area of research for ubiquitous display interaction (e.g., [1,19,20]) These techniques enable users to quickly and seamlessly interact with nearby displays, and have many practical advantages. To name only a few: proprioperceptive interaction allows one to avoid focusing on the physical input device; hand and arm gestures can naturally combine intended target and command into a seamless whole; given gestures' role in interpersonal communication, there is some consensus on gesture meaning.…”