Multi-moji

Abstract

This paper explores the combination of multiple concurrent modalities for conveying emotional information in HCI: temperature, vibration and abstract visual displays. Each modality has been studied individually, but can only convey a limited range of emotions within two-dimensional valencearousal space. This paper is the first to systematically combine multiple modalities to expand the available affective range. Three studies were conducted: Study 1 measured the emotionality of vibrotactile feedback by itself; Study 2 measured the perceived emotional content of three bimodal combinations: vibrotactile + thermal, vibrotactile + visual and visual + thermal. Study 3 then combined all three modalities. Results show that combining modalities increases the available range of emotional states, particularly in the problematic top-right and bottom-left quadrants of the dimensional model. We also provide a novel lookup resource for designers to identify stimuli to convey a range of emotions.

Touch without Touching: Overcoming Social Distancing in Semi-Intimate Relationships with SansTouch

“…Most touch devices used vibration motors to generate the touch stimuli. For example, Multi-Moji [50] used the built-in vibration motors in smartphones to enhance emojis with tactile feedback. TaSSt [24] and VisualTouch [53] used vibration motor arrays embedded on a tactile sleeve to generate tactile patterns on forearms.…”

“…Combining touch with other modalities could also expand the available range of emotional states and increase the touch perception [43,50,53]. For example, Wilson et al [50] showed that combining visual and thermal modalities in touch interactions allowed users to convey a wider range of afective expressions.…”

“…Combining touch with other modalities could also expand the available range of emotional states and increase the touch perception [43,50,53]. For example, Wilson et al [50] showed that combining visual and thermal modalities in touch interactions allowed users to convey a wider range of afective expressions. Tewell et al [43] suggested that by adding a thermal array display (TAD) to text messaging systems, the emotional perception of the message could be enhanced.…”

“…Previous works suggested that adding visual modality on remote touch devices could efectively enhance tactile perception [50,53]. Hence, we added one layer of addressable RGB LED (ref.…”

“…Based on these insights, we built SansTouch, a multimodal hand sleeve used along with a smartphone to enable handto-hand touch interactions without directly touching. SansTouch combined three modalities that were recommended in the literature to enhance touch sensations: skin-like infation [31,45], heat conditioned to body temperature [3,17,31], and visual feedback [15,50,53]. We then conducted a structured observational study with 12 participants to 1) evaluate user perceptions and the use of SansTouch's handshakes in a face-to-face communication for semiintimate relationships (i.e., colleagues); and 2) elicit user feedback regarding the generalizability aspects of SansTouch, including the use of SansTouch's hand-to-hand interactions for other types of relationships and remote communications.…”