Automatic configuration of spatially consistent mouse pointer navigation in multi-display environments

Abstract

Multi-display environments combine displays of various form factors into a common interaction space. Cross-display navigation techniques have to provide transitions to move the mouse pointer across display boundaries to reach distant display locations. A spatially consistent description of display relationships thereby supports fluid cross-display navigation. In this paper, we present two spatially consistent navigation techniques for seamless cross-display navigation in multi-user multi-display environments. These navigation techniques are automatically configured from a spatial model of the environment, which is generated in a cameraassisted calibration step. We describe the implementation in a distributed system and present results of a comparative experiment.

Cross-Device Taxonomy

“…For example, using on-screen interactions, techniques such as stitching [129] or pinching [192,238,243] multiple display together, or performing synchronous tapping touches [280] have been used to pair devices into one configuration. Pairing techniques have also leveraged on-device pointing [259,349] or different finger posture and identification [144] to implicitly create cross-device configurations. For around-thedevice interactions, examples include knocking to pair [103], or even taking a picture to recognise other devices [299].…”

“…Pair, select or engage with device by: → stitching [54,129] → pinching [190,192,238,243] → synchronous tapping [280] → vision-based handshake [360] → pointing [259,349] → finger postures [141] → painting on surfaces [370] Pair, select or engage with device by: → touching the surface [293] → knocking or tapping [103,280] → tapping appliance [372] → taking picture to recognize device [299] → roll-and-pointing [61] → hold-and-flipping [54] → pitching-to-open [54] Pair or engage with device by: → bumping [127] → synchronous gestures [272] → stacking [61] → snapping a picture pair [60] → shaking [131,211] → placing down device [143,270,360] → recognizing motion correlation [359] Pair or engage with device by: → bending [61] → using sandwich structure [61] → stretching [61] Pair or engage with device by: → approaching [12,203] → turning body towards [12,107,205,369] → detecting presence of person [12,30,145,369] → detect groups…”

“…→Space Window Manager [24] and Layout Manager [357] →World-in-miniature [24,323,358] →RadarMap [2,97], Map [24], MiniMap [16] 2 | Content visualisation and distribution strategies →Multiple coordinated views [48,91,92,120,263,322] →Brush-and-linking, across devices [7,63] →Overview and detail [16,136,287,322] →Central overview device [31] →Focus and Context [98,322] →Adaptive views, applications, adaptive UIs [21,150,373] →Overlaying information [357] →MagicLens views [16,28] →Dynamically corrected perspective views [226] 3 | Feedback and feedforward approaches →Object shadow [22] →Gesture shadows/ghosts [22,188,301] →Temporary preview shadows of content [206] →Perspective cursor [227,349] →Multi-display pointers [163,286]…”

“…Third, to increase the intelligibility of crossdevice systems, feedback and feedforward mechanisms have been developed. Examples such as object preview and gesture shadows [22,188,206,301], perspective cursor [227,349] or multi-display pointers [163,286] help users understand how information is travelling between devices. A final strategy is in explicitly visualising the links between devices.…”

Smart Interaction Management: An Interaction Approach for Smart Meeting Rooms

“…More approaches are dealing with interaction pointer movement in heterogeneous multi-display environments (Perspective Cursor [11], Deskotheque [12], etc. ).…”