Occlusion-aware interfaces

Abstract

We define occlusion-aware interfaces as interaction techniques which know what area of the display is currently occluded, and use this knowledge to counteract potential problems and/or utilize the hidden area. As a case study, we describe the Occlusion-Aware Viewer, which identifies important regions hidden beneath the hand and displays them in a non-occluded area using a bubble-like callout. To determine what is important, we use an application agnostic image processing layer. For the occluded area, we use a user configurable, real-time version of Vogel et al. 's [21] geometric model. In an evaluation with a simultaneous monitoring task, we find the technique can successfully mitigate the effects of occlusion, although issues with ambiguity and stability suggest further refinements. Finally, we present designs for three other occlusion-aware techniques for pop-ups, dragging, and a hidden widget.

Evaluating Physical/Virtual Occlusion Management Techniques for Horizontal Displays

“…We do not claim that the above six techniques are exhaustive examples of the design space of physical occlusion management; additional possibilities include object indices, distortion, and labels, etc. We also do not claim that these techniques are novel; in fact, some explicitly derive from existing techniques such as Shift [7], hybrid piles [22], and occlusion-aware interfaces [6]. However, we do think these techniques are a representative sampling of the occlusion management design space.…”

“…More advanced techniques make windows semi-transparent [3], cut holes in them to expose hidden items [4], or spread them out to make hidden content accessible [5]. Other work considers situations when the user's hand and arm occlude the display on touch and pen-based systems [6][7][8]. However, with the exception of contextual display bubbles [9] and relations between virtual and digital media [2], little work has been performed on managing occlusion between virtual and physical objects beyond the user's own body.…”

“…This is similar to Shift [7], where a copy of the occluded region is moved to a non-occluded region. Building on this, more recent work on occlusion-aware interfaces [6] models which area is occluded and uses this knowledge to display hidden objects in a non-occluded screen area.…”

“…• Replicate (REPL): Inspired by Vogel et al [6], this technique replicates the entire screen content of a particular physical occluder in a callout located in unoccluded space as close to the occluder as possible (Figure 2(f)). Objects are revealed in their natural size and their relative position is maintained.…”

“…We argue that these tasks are building blocks for higher-level tasks involving more complex objects like text documents and applications, so our results should therefore generalize even to such objects. − Static occlusion: Existing work [6,8] on physical occlusion between a user's arm and the display deals with intrinsically dynamic situations with moving targets and occluders. However, the primary problem on tabletops is static occlusion caused by physical objects placed on the table surface.…”