Isolating shape from semantics in haptic-visual priming

Abstract

The exploration of a familiar object by hand can benefit its identification by eye. What is unclear is how much this multisensory cross-talk reflects shared shape representations versus generic semantic associations. Here, we compare several simultaneous priming conditions to isolate the potential contributions of shape and semantics in haptic-to-visual priming. Participants explored a familiar object manually (haptic prime) while trying to name a visual object that was gradually revealed in increments of spatial resolution. Shape priming was isolated in a comparison of identity priming (shared semantic category and shape) with category priming (same category, but different shapes). Semantic priming was indexed by the comparisons of category priming with unrelated haptic primes. The results showed that both factors mediated priming, but that their relative weights depended on the reliability of the visual information. Semantic priming dominated in Experiment 1, when participants were free to use high-resolution visual information, but shape priming played a stronger role in Experiment 2, when participants were forced to respond with less reliable visual information. These results support the structural description hypothesis of haptic-visual priming (Reales and Ballesteros in J Exp Psychol Learn Mem Cogn 25:644-663, 1999) and are also consistent with the optimal integration theory (Ernst and Banks in Nature 415:429-433, 2002), which proposes a close coupling between the reliability of sensory signals and their weight in decision making.

Multisensory interaction in product choice: Grasping a product affects choice of other seen products

“…The right container was named significantly more often as first object by participants in the visuo‐haptic group (54.3%) than by those in the visual‐only group (22.9%), Pearson χ 2 (1) = 4.34, p < .05. This result conceptually replicates prior demonstrations of visuo‐haptic priming (Pesquita et al, 2013; Reales & Ballesteros, 1999). Grasping an object facilitates visual perception of other products of the same shape and size.…”

“…Another novel contribution is to demonstrate the compensatory relationship between vision and touch in product evaluation. When visual information is somehow degraded (e.g., by decreasing clarity), then haptic information assumes a greater role in object recognition (Ernst & Banks, 2002; Helbig & Ernst, 2007; Pesquita et al, 2013). Consequently, we showed for the first time that the effect of touch on product choice is accentuated by crowded product displays, which may overload the visual system and hence increase reliance on haptic information.…”

“…For instance, the visual priming of contours increases liking of objects with a matching shape because the object is perceived more fluently (Winkielman & Cacioppo, 2001). And as reviewed above, haptic shape perception can also facilitate visual processing of that shape (e.g., Pesquita et al, 2013). Critically however, we predicted that haptic exposure to a source product may increase the visual fluency of a target product that is haptically similar but spatially distant (cf.…”

“…Consequently, purely haptic exposure to a familiar object (i.e., manually exploring without viewing) facilitates visual recognition of the given object (Reales & Ballesteros, 1999). For example, after manually exploring a spoon, people are faster to recognize an image of a spoon (Pesquita, Brennan, Enns, & Soto-Faraco, 2013). However, it remains unclear whether such visuo-haptic priming also occurs when the haptic and visual signals arise from clearly distinct sources (Gephstein, Burge, Ernst, & Banks, 2005).…”

Not so automatic: Task relevance and perceptual load modulate cross-modal semantic congruence effects on spatial orienting

“…It is therefore possible that in the natural environment object-based (semantic) relations between sounds and visual events might have an influence on attention orienting. Several recent studies have addressed the role of crossmodal semantic congruence on spatial orienting by investigating how characteristic sounds of objects (musical instruments, vehicles, animals etc) or semantically congruent tactile information can enhance performance in different visual tasks (e.g., Laurienti et al, 2004;Chen and Spence, 2011;Molholm et al, 2004;Pesquita et al, 2013;Iordanescu et al, 2008;Iordanescu et al, 2010;List et al, 2014). However, the results of these studies are mixed, some suggesting that semantic congruence effectively attracts attention and some suggesting that it does not.…”