Selection Performance Using a Smartphone in VR with Redirected Input

Abstract

We present a method to track a smartphone in VR using a fiducial marker displayed on the screen. Using WebRTC transmission protocol, we capture input on the smartphone touchscreen as well as the screen contents, copying them to a virtual representation in VR. We present two Fitts' law experiments assessing the performance of selecting targets displayed on the virtual smartphone screen using this method. The first compares direct vs. indirect input (i.e., virtual smartphone co-located with the physical smartphone, or not), and reveals there is no significant difference in performance due to input indirection. The second experiment assesses the influence of input scaling, i.e., decoupling the virtual cursor from the actual finger position on the smartphone screen so as to provide a larger virtual tactile surface. Results indicate a small effect for extreme scale factors. We discuss implications for the use of smartphones as input devices in VR.