Blended Interaction: understanding natural human–computer interaction in post-WIMP interactive spaces

Abstract

We introduce Blended Interaction, a new conceptual framework that helps to explain when users perceive user interfaces as ''natural'' or not. Based on recent findings from embodied cognition and cognitive linguistics, Blended Interaction provides a novel and more accurate description of the nature of human computer interaction (HCI). In particular, it introduces the notion of conceptual blends to explain how users rely on familiar and real-world concepts whenever they learn to use new digital technologies. We apply Blended Interaction in the context of post-''Windows Icons Menu Pointer'' interactive spaces. These spaces are ubiquitous computing environments for computer-supported collaboration of multiple users in a physical space or room, e.g., meeting rooms, design studios, or libraries, augmented with novel interactive technologies and digital computation, e.g., multi-touch walls, tabletops, and tablets. Ideally, in these spaces, the virtues of the familiar physical and social world are combined with that of the digital realm in a considered manner so that desired properties of each are preserved and a seemingly ''natural'' HCI is achieved. To support designers in this goal, we explain how the users' conceptual systems use blends to tie together familiar concepts with the novel powers of digital computation. Furthermore, we introduce four domains of design to structure the underlying problem and design space: individual and social interaction, workflow, and physical environment. We introduce our framework by discussing related work, e.g., metaphors, mental models, direct manipulation, image schemas, reality-based interaction, and illustrate Blended Interaction using design decisions we made in recent projects.

User Experience in Cross-channel Ecosystems

“…The concept of blending has also been used for the design of ambient assisted living environments for the elderly (Hoshi and Waterworth, 2009) and for the design of a novel table lamp (Wang, 2016). Blending theory has also been used in the context of blended interaction (Jetter, Reiterer, and Geyer, 2014) where the aim is to bring the experiences of real world interactions to user interface design and in other approaches to HCI and software engineering (Imaz and Benyon, 2007).…”

Studying the benefits and challenges of spatial distribution and physical affordances in a multi-device workspace

“…This led to a new research domain addressing multi-device settings with the goal that individual devices should no longer act like small isolated units but should form an ecology that allows for easy combinations, relations and transfers among them [16]. The multitude of available devices encourages the allocation of tasks to specific devices [12] following the concept of traditional tools such as pens or a ruler.…”

Information Visualization Analysis of Human-computer Interaction Interface

“…When the users are carrying out the mining in the visualized interface that presents largescale data, their cognitive psychology tends to have exploration on the local detailed information that they are interested in, and at the same time, keep the visibility of the overview of the information space at any time (Area, Coquet, Ortega, López and Rodríguez-Ascariz, 2012). The interactive technology that has been developed based on this cognitive law mainly includes two categories of Overview+Detail and Focus+Context (Better, Litterer and Beyer, 2014). The Overview+Detail technology divides the information space into two views, which respectively provides the overview that is applied to the global navigation of the information space, and the local detail information view.…”