Multimodal perception of material properties

Martín, Rodrigo; Iseringhausen, Julian; Weinmann, Michael; Hullin, Matthias B.

doi:10.1145/2804408.2804420

Cited by 15 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in reality, humans often use multiple senses to judge material properties of objects. Previous literature has shown that inputs from multiple senses often interact during material perception (Bonneel, Suied, Viaud-Delmon, & Drettakis, 2010;Buckingham, Cant, & Goodale, 2009;Fujisaki, Goda, Motoyoshi, Komatsu, & Nishida, 2014;Fujisaki, Tokita, & Kariya, 2015;Martín, Iseringhausen, Weinmann, & Hullin, 2015;Tiest & Kappers, 2007). The majority of multisensory studies of human material perception have focused on measuring one or certain specific attributes, such as surface roughness (Tiest & Kappers, 2007).…”

Section: Introductionmentioning

confidence: 99%

Can you see what you feel? Color and folding properties affect visual–tactile material discrimination of fabrics

Xiao

Wei

et al. 2016

Journal of Vision

View full text Add to dashboard Cite

Humans can often estimate tactile properties of objects from vision alone. For example, during online shopping, we can often infer material properties of clothing from images and judge how the material would feel against our skin. What visual information is important for tactile perception? Previous studies in material perception have focused on measuring surface appearance, such as gloss and roughness, and using verbal reports of material attributes and categories. However, in real life, predicting tactile properties of an object might not require accurate verbal descriptions of its surface attributes or categories. In this paper, we use tactile perception as ground truth to measure visual material perception. Using fabrics as our stimuli, we measure how observers match what they see (photographs of fabric samples) with what they feel (physical fabric samples). The data shows that color has a significant main effect in that removing color significantly reduces accuracy, especially when the images contain 3-D folds. We also find that images of draped fabrics, which revealed 3-D shape information, achieved better matching accuracy than images with flattened fabrics. The data shows a strong interaction between color and folding conditions on matching accuracy, suggesting that, in 3-D folding conditions, the visual system takes advantage of chromatic gradients to infer tactile properties but not in flattened conditions. Together, using a visual-tactile matching task, we show that humans use folding and color information in matching the visual and tactile properties of fabrics.

show abstract

Section: Introductionmentioning

confidence: 99%

Can you see what you feel? Color and folding properties affect visual–tactile material discrimination of fabrics

Xiao

Wei

et al. 2016

Journal of Vision

View full text Add to dashboard Cite

show abstract

“…For the second activity, participants were able to interact in a free-way. At the end of each condition, participants answered the User Experience Questionnaire (UEQ) 1 [22] and the sensory experience questionnaire 2 [19,24]. To compute the questionnaire scores, we scored each item using a scale from -3 to +3, and we calculated the average score for all the items.…”

Section: Pilot Study: Potential Differences Between Rigid and Elastic Displaysmentioning

confidence: 99%

Interactions of children and young adults using large-scale elastic displays

et al. 2020

View full text Add to dashboard Cite

Elastic displays provide a unique and intuitive interaction and could be deployed at large-scale. As an emerging technology, open questions about the benefits large-scale elastic displays offer over rigid displays and their potential application to our everyday lives. In this paper, we present an overview of a 4-year project. First, we describe the development of a large-scale elastic display, called BendableSound. Second, we explain the results of a laboratory study, showing the elastic display has a better user and sensory experience than a rigid one. Third, we describe the results of two deployment studies showing how BendableSound could support the therapeutic practices of children with autism and the early development of toddlers. We close discussing open challenges to study the untapped potential of elastic displays in pervasive computing.

show abstract

“…A high degree of consistency between these two assignments was detected, indicating that they facilitate one another by accessing the same perceptual information. The multisensory nature of the communication of material qualities has been further explored by Martín et al [18], where the authors employed contact and stroking material sounds to complement the visual stimuli. Their results demonstrate the strong linkage between the auditory channel and the haptic perception to a point in which sound is capable of biasing the visual judgment of concrete qualities.…”

Section: Related Workmentioning

confidence: 99%

“…As a logical consequence is desirable to enhance the digital material with additional cues on top of the purely visual user experience. In this regard, auditory cues and sonification techniques have demonstrated to influence significantly the perception of a © 2018 Copyright held by the owner/author (s) product quality/efficiency [22], compensate the absence of tactile interaction with digital material samples [18] and increase the feeling of immersion w.r.t. the unimodal visual experience [15].…”

Section: Introductionmentioning

confidence: 99%

A Study of Material Sonification in Touchscreen Devices

Martín

Weinmann

Hullin

2018

Proceedings of the 2018 ACM International Conference on Interactive Surfaces and Spaces

Self Cite

View full text Add to dashboard Cite

Even in the digital age, designers largely rely on physical material samples to illustrate their products, as existing visual representations fail to sufficiently reproduce the look and feel of real world materials. Here, we investigate the use of interactive material sonification as an additional sensory modality for communicating well-established material qualities like softness, pleasantness or value. We developed a custom application for touchscreen devices that receives tactile input and translate it into material rubbing sound using granular synthesis. We used this system to perform a psychophysical study, in which the ability of the user to rate subjective material qualities is evaluated, with the actual material samples serving as reference stimulus. Our experimental results indicate that the considered audio cues do not significantly contribute to the perception of material qualities but are able to increase the level of immersion when interacting with digital samples.

show abstract

Multimodal perception of material properties

Cited by 15 publications

References 24 publications

Can you see what you feel? Color and folding properties affect visual–tactile material discrimination of fabrics

Can you see what you feel? Color and folding properties affect visual–tactile material discrimination of fabrics

Interactions of children and young adults using large-scale elastic displays

A Study of Material Sonification in Touchscreen Devices

Contact Info

Product

Resources

About