Sublimate

Abstract

Recent research in 3D user interfaces pushes towards immersive graphics and actuated shape displays. Our work explores the hybrid of these directions, and we introduce sublimation and deposition, as metaphors for the transitions between physical and virtual states. We discuss how digital models, handles and controls can be interacted with as virtual 3D graphics or dynamic physical shapes, and how user interfaces can rapidly and fluidly switch between those representations. To explore this space, we developed two systems that integrate actuated shape displays and augmented reality (AR) for co-located physical shapes and 3D graphics. Our spatial optical see-through display provides a single user with head-tracked stereoscopic augmentation, whereas our handheld devices enable multi-user interaction through video seethrough AR. We describe interaction techniques and applications that explore 3D interaction for these new modalities. We conclude by discussing the results from a user study that show how freehand interaction with physical shape displays and co-located graphics can outperform wand-based interaction with virtual 3D graphics.

Tailored Controls

“…Complex tangible interfaces that feature high representation capabilities are enabled by the interface changing its shape [11]. This may be combined with an augmented reality (AR) application [24] to add a tangible representation of the AR content. Shape-changing tangible interfaces can also be used to combine different elements such as sliders or knobs in a single component [21].…”

Morphees+

“…For instance, Length is good to describe actuated pin displays [11,14,53,29,31,33]. In inFORM [11], each pin can extend 100 mm.…”

Combining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance

“…Shape-changing interfaces provide benefits with their ability to dynamically alter the physical shape of objects. They allow for dynamic tactile sensation (e. g. [41]), haptic data exploration (e. g. [26]) or in general diverse dynamic affordances (e. g. [23]). However, changing the physical shape of devices to resemble desired target shapes can be challenging.…”

“…Additionally, even if fast physical motion is possible, it can be problematic since it could startle users (cf. [26]). While research on shape-changing interfaces focused mostly on altering objects physically, a wide range of prior work aimed at changing the optical appearance of objects and environments, for example through spatial augmented reality (spatial AR, e. g. [7,31,21,24]) or fish tank VR (e. g. [36]).…”

“…Until now, most research has focused on either physical or visual change. In the field of shape-changing interfaces, devices showed co-located 2D information (e. g. MorePhone [19]), were augmented with 2.5D height field data (e. g. Relief [27]), or extended with see-through augmented reality (e. g. Sublimate [26]). None of these approaches extends shape-changing interfaces with perspective-corrected 3D visuals in a physically co-located way.…”