Effects of Facial Expression and Language on Trustworthiness and Brain Activities

Abstract

Social communication uses verbal and nonverbal language. We examined the degree of trust and brain activity when verbal and facial expressions are incongruent. Fourteen healthy volunteers viewed photographs of 8 people with pleasant (smile) or unpleasant expressions (disgust) alone or combined with a verbal [positive/negative] expression. As an index for degree of trust, subjects were asked to offer a donation when told that the person in the photograph was troubled financially. Positive emotions and degree of trust were evaluated using the Visual Analogue Scale (VAS). Event-related potentials (ERPs) were obtained at 170-240 ms after viewing the photographs. Brain activity during incongruent conditions was localized using standardized Low Resolution Brain Electromagnetic Tomography (sLORETA). VAS scores for positive × smile condition were significantly higher than those for the other conditions ( < 0.05). The donation offered was significantly lower for incongruence between verbal and facial expressions, particularly for negative × smile condition. EEG showed more activity in the parietal lobe with incongruent than with congruent conditions. Incongruence [negative × smile] elicited the least positive emotion, degree of trust, and amount of offer. Our results indicate that incongruent sensory information increased activity in the parietal lobe, which may be a basis of mentalizing.

Incongruence between Verbal and Non-Verbal Information Enhances the Late Positive Potential

“…The donation task was formulated based on a previous study [ 4 ] demonstrating that people paid more money to individuals who were rated subjectively as having a more trustworthy face. Additionally, in our previous study, we showed that people donated more money to a more trustworthy person [ 7 ]. Thus, the present donation task has been shown to be valid to estimate trustworthiness.…”

“…In our previous study, we used low resolution brain electromagnetic tomography (LORETA) to investigate the brain activity of individuals receiving stimuli consisting of positive or negative verbal information paired with a smiling face, as well as positive or negative verbal information paired with an expression of disgust. Our results suggest that activity in the parietal lobe increases when observing a smiling face paired with negative verbal information, that is, when the verbal information and facial expression are incongruent [ 7 ]. Therefore, when the verbal information and facial expression information were incongruent, activity was demonstrated not only in the anterior cingulate gyrus and the prefrontal area, but also in the parietal lobes, which are also involved in the information interference effect.…”