Making sense of digitally remediated touch in virtual reality experiences

Editorial

“…Understanding the physical, material, and sensory aspects of touch is, we argue, a central part of when and how touch-based resources are taken up (or excluded) and how they can shape –or are shaped by – people to become semiotic resources. We have shown for example, that the physical and material qualities of hand-held virtual controllers, and sensory expectations of touch built through prior experience are central to a sense of touch in virtual reality environments (Jewitt et al, 2021). Multimodality seeks to map the semiotic resources and affordances of touch, the qualities of touch and their experiences and associations, and the meaning potentials these represent, as a descriptive inventory of the resources and types of touch made available in a given context.…”

“…We have sought to map the features, materiality and semiotic potential of touch, and to ask under what social conditions and in what social contexts are touch-based resources shaped through their use by people to become semiotic resources or modes, as well as what people use touch to achieve and the established conventions that inform their use. For example, we have used a multimodal framework to explore what is counted as touch in virtual reality encounters (Jewitt et al, 2021; Price et al, 2021). We have also explored the use of digitally mediated touch to communicate intimacy and reassurance between friends and family, and what meanings appear to be associated with the dimensions of touch (location, duration, or pressure), and how these are used (Price et al, 2022).…”

“…The search for new methods to engage with the complexity of touch has intensified multimodal interest in mobilizing both sensory and artistic research methods (Jewitt et al, 2021; Jewitt et al, 2022). Exploiting methodological synergies across these frontiers can, we argue, open multimodality up to different perspectives, generate imaginative research questions and make available a wider range of methodological tools for creative use.…”

Taking an Extended Embodied Perspective of Touch: Connection-Disconnection in iVR

“…While there are several constraints, touch in iVR can "feel real" even if the physical experience is very degraded (Parisi, in Candy, 2019;Jewitt et al, 2021), especially given that iVR engages with the sociocultural and affective elements of touch through powerful multisensory (visual, audio, haptic) and contextual cues. The importance of illusion arises since here touch becomes a kind of "imagined" experience, where the brain is tricked to sense the experience through exploitation of perceptual gaps and provision of appropriate multisensory stimuli (Biocca et al, 2001).…”

“…The notion of touch in the literature embraces a broader more "embodied" idea of touch experience (e.g., Sheets-Johnstone, 1999;Paterson, 2006;Parisi, 2014), where touch is conceived as extending beyond the point of contact (that one might have with the hand on the controller), and brings attention to the whole body-in-movement with its ways of touching and "feeling." The relevance to expanding the sense of touch through the lens of embodiment (as elaborated on in An Extended Embodied Sense of Touch) is timely given the growing awareness that: (1) users engagement with, and experiences of, virtual realities are shaped by the histories, expectations, subjectivities they bring with them (Hollett et al, 2019;Jewitt et al, 2021); (2) recreating touch is technologically complex, not least because touch consists of more than cutaneous events, it is mobile and distributed throughout the body (Parisi, 2014); and (3) touch is felt deeper than the skin where, through phenomenological investigations into embodiment, complex but intimate relationships between touching and feelings of connectedness (Paterson, 2009) are exposed and where the relationships between touch and movement are pivotal to how we come to feel the world (Sheets-Johnstone, 1999).…”