What do you do when two hands are not enough? interactive selection of bonds between pairs of tangible molecules

Abstract

For molecular modeling, chemical structures have to be understood and imagined both in their three-dimensional spatial extent and in their dynamic behavior. We have developed an AR-based system for tangible interaction with molecules using optical markers. When users bring several molecules close to one another, potential bonds are shown and the molecules dynamically change their 3D structure according to potential chemical reactions. A problem arises when users also need to select one such bond from of a multitude of potential bonds while already using both hands to manipulate the molecules. We present two gesture-based techniques, shake-based and proximity-based to solve this problem. We report on user tests evaluating these techniques with respect to speed, precision and user acceptance.

Augmented Chemical Reactions: 3D Interaction Methods for Chemistry

“…The possible binding partners of each molecule were subdivided into several groups which could be activated by shaking the molecule. After selecting this subgroup of possible binding partners, the proximity method can be used to finally select the desired possible bond [7].…”

Designing and comparing two-handed gestures to confirm links between user controlled objects

“…The project and thus our system also provides a tool that enable users to build molecules from atoms and reactants and to observe the dynamic spatial relations of binding * e-mail: maierp@in.tum.de, toennis@in.tum.de, klinker@in.tum.de possibilities. When multiple open bonds are present, the system provides techniques for selecting any of these bonds [5].…”

On the Usability and Effectiveness of Different Interaction Types in Augmented Reality

“…Y. Lee, Rhee, & Seo, 2010;Maier et al, 2010;Marco et al, 2012;Nagel & Heidmann, 2011;Ness et al, 2010;Radkowski & Stritzke, 2012;White, Feng, & Feiner, 2009;Xu et al, 2007). First-Person Vision leveraged by mobile video cameras is currently considered the most optimal way to sense an environment and a subject's activities from a wearable sensor, as indicated by Kanade and Hebert (2012).…”