SurfaceLink

Abstract

We present SurfaceLink, a system where users can make natural surface gestures to control association and information transfer among a set of devices placed on a mutually shared surface (e.g., a table). SurfaceLink uses a combination of on-device accelerometers, vibration motors, speakers, and microphones (and, optionally, an off-device contact microphone for greater sensitivity) to sense gestures performed on the shared surface. In a controlled evaluation with 10 participants, SurfaceLink detected the presence of devices on the same surface with 97.7% accuracy, their relative arrangement with 89.4% accuracy, and a set of single-and multi-touch surface gestures with an average accuracy of 90.3%. A usability analysis showed that SurfaceLink has advantages over current multi-device interaction techniques in a number of situations.

Cross-Device Taxonomy

“…How to use the tracking classification: Selecting an appropriate cross-device tracking technology is a challenging task -even for experts in the field. The choice follows several considerations weighing the benefits of outside-in tracking that provides high fidelity information as opposed to a more Distance without cross-device synchronization [258] , motion-resistant distance calculation [378] Relative 2D positioning using custom ultrasound dongles [123,97] Absolute 3D positioning on unmodified mobile phones [159] , conceptual relative 3D positioning on static mobile phones [267] , [159] user-generated Doppler-based gesture recognition between 2 phones [50] , Doppler-based multi-device selection [6] , swiping between devices on surface [103] , relative positioning from ambient sounds [327] , positioning from ambient sounds [137] capacitive capacitive…”

“…Pairing techniques have also leveraged on-device pointing [259,349] or different finger posture and identification [144] to implicitly create cross-device configurations. For around-thedevice interactions, examples include knocking to pair [103], or even taking a picture to recognise other devices [299].…”

“…Pair, select or engage with device by: → stitching [54,129] → pinching [190,192,238,243] → synchronous tapping [280] → vision-based handshake [360] → pointing [259,349] → finger postures [141] → painting on surfaces [370] Pair, select or engage with device by: → touching the surface [293] → knocking or tapping [103,280] → tapping appliance [372] → taking picture to recognize device [299] → roll-and-pointing [61] → hold-and-flipping [54] → pitching-to-open [54] Pair or engage with device by: → bumping [127] → synchronous gestures [272] → stacking [61] → snapping a picture pair [60] → shaking [131,211] → placing down device [143,270,360] → recognizing motion correlation [359] Pair or engage with device by: → bending [61] → using sandwich structure [61] → stretching [61] Pair or engage with device by: → approaching [12,203] → turning body towards [12,107,205,369] → detecting presence of person [12,30,145,369] → detect groups…”

“…development framework, toolkit, middleware) [124,145,152,174,193,235,243,304,328,373] → of an interaction technique [54,66,94,118,147,186,300,322,374] → of a theoretical framework; constructiveconceptual [245,316,324] → focus groups and workshops [59,158,179,245,246,283,306] → design sessions and co-creation [234,283] → other informal & early demonstrations [128,138,279,303,363] Technical evaluation (66) → performance, compared to other systems [159,227,269,320,333] → quality measurements (e.g. accuracy of tracking) [6,7,47,54,55,103,…”

“…time, frequency, FPS, round trip time, memory, etc.) [6,47,103,255,258,270,282,320,339,378] Heuristic evaluation (5) → as only evaluation method [173,174] → together with qualitative user studies [197,353] → together with case studies [304] → qualitative user study based on heuristic criteria [236] physical and digital coloured borders are used to indicate connectivity [116,206,271,351,358]. Visual proxies or portals on or around devices are used to visualise what device owns which information [24,64,89,97,123,143,203,357,358], and cross-device portals help users identify the boundaries between devices [143,206].…”