Can you see what you feel? Color and folding properties affect visual–tactile material discrimination of fabrics

Abstract

Humans can often estimate tactile properties of objects from vision alone. For example, during online shopping, we can often infer material properties of clothing from images and judge how the material would feel against our skin. What visual information is important for tactile perception? Previous studies in material perception have focused on measuring surface appearance, such as gloss and roughness, and using verbal reports of material attributes and categories. However, in real life, predicting tactile properties of an object might not require accurate verbal descriptions of its surface attributes or categories. In this paper, we use tactile perception as ground truth to measure visual material perception. Using fabrics as our stimuli, we measure how observers match what they see (photographs of fabric samples) with what they feel (physical fabric samples). The data shows that color has a significant main effect in that removing color significantly reduces accuracy, especially when the images contain 3-D folds. We also find that images of draped fabrics, which revealed 3-D shape information, achieved better matching accuracy than images with flattened fabrics. The data shows a strong interaction between color and folding conditions on matching accuracy, suggesting that, in 3-D folding conditions, the visual system takes advantage of chromatic gradients to infer tactile properties but not in flattened conditions. Together, using a visual-tactile matching task, we show that humans use folding and color information in matching the visual and tactile properties of fabrics.

Shatter and splatter: The contribution of mechanical and optical properties to the perception of soft and hard breaking materials

“…The approach that we adopted in this study was to ask observers to directly judge or rate different attributes such as glossiness and transparency, which are usually considered to be directly perceivable, or fluffiness and airiness, which are more inferred (i.e., derived from associations with shape, motion, and/or optical appearance). This approach has been criticized by some researchers for relying on observers' ability to use language to describe material properties (e.g., Xiao, Bi, Jia, Wei, & Adelson, 2016). We do not disagree with this view; however, we believe that in the present context such an approach was both appropriate and very informative.…”

Estimating mechanical properties of cloth from videos using dense motion trajectories: human psychophysics and machine learning

“…There have been a few recent works addressing the influence of optical properties on material perception, such as viscosity of liquids and classification of cloth (Aliaga et al, 2015;Paulun et al, 2015;Xiao, Bi, Jia, Wei, & Adelson, 2016;Assen & Fleming, 2016). Our goal was to evaluate the role of multi-frame motion in the perception of mechanical properties, but this does not exclude the role of appearance.…”

Dynamic dot displays reveal material motion network in the human brain

“…Literature investigating neural mechanisms of material perception has so far heavily focused on how the brain discriminates between different optical appearances (Komatsu & Goda, 2018). Non-optical properties, however, can also be conveyed through images: we can infer the 'feel' of soft silk or fur just by looking at an image of it (Baumgartner et al 2013, Xiao et al 2016 owing to previously formed associations between the different senses when interacting with materials; over time, specific visual (or auditory, proprioceptive, or olfactory) information becomes associated with specific tactile information and vice versa. It is possible that via this indirect route (Schmidt et al 2017) mechanical and tactile material qualities like softness, viscosity, and roughness can be conveyed through optical properties of surfaces and the 3D structure of visual objects (e.g.…”

“…It is possible that via this indirect route (Schmidt et al 2017) mechanical and tactile material qualities like softness, viscosity, and roughness can be conveyed through optical properties of surfaces and the 3D structure of visual objects (e.g. Baumgartner et al 2013;Fleming, 2014Fleming, , 2017Fleming et al 2013;Giesel & Zaidi, 2013;Ho et al 2006;van Assen & Fleming 2016;Xiao et al 2016).…”