Semantic processing of self-adaptors, emblems, and iconic gestures: An ERP study

Chui, Kawai; Lee, Chia‐Ying; Yeh, Kanyu; Chao, Pei-Chun

doi:10.1016/j.jneuroling.2018.04.004

Cited by 17 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We appreciate that further research is required to help understand the cognitive mechanism that underpins the Mental Priming Force, but we believe that it relies on semantic priming. Several studies have shown that people process specific gestures semantically (37)(38)(39) and it is likely that they evoke similar semantic activation that is found for words or pictures (40). We therefore suggest that the Mental Priming Force relies on gestures and speech segments evoking simple semantic activation that make the number 3 and diamond shape more accessible.…”

Section: Discussionsupporting

confidence: 52%

Influencing choices with conversational primes: How a magic trick unconsciously influences card choices

Pailhès

Kuhn

2020

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Past research demonstrates that unconscious primes can affect people’s decisions. However, these free choice priming paradigms present participants with very few alternatives. Magicians’ forcing techniques provide a powerful tool to investigate how natural implicit primes can unconsciously influence decisions with multiple alternatives. We used video and live performances of the mental priming force. This technique uses subtle nonverbal and verbal conversational primes to influence spectators to choose the three of diamonds. Our results show that a large number of participants chose the target card while reporting feeling free and in control of their choice. Even when they were influenced by the primes, participants typically failed to give the reason for their choice. These results show that naturally embedding primes within a person’s speech and gestures effectively influenced people’s decision making. This raises the possibility that this form of mind control could be used to effectively manipulate other mental processes.

show abstract

Section: Discussionsupporting

confidence: 52%

Influencing choices with conversational primes: How a magic trick unconsciously influences card choices

Pailhès

Kuhn

2020

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…More generally, their research shows that disfluencies in speech are mirrored by disfluencies in gestures. In this sense, most studies tend to observe how visual cues are generated along with or close to language (Chui et al, 2018;Hoek et al, 2017). Most of them address non-verbal elements by mapping them to linguistic forms (Lin, 2017), i.e., synching their occurrence with the corresponding lexical form (e.g., making cutting gestures when uttering "cut").…”

Section: Related Workmentioning

confidence: 99%

Understanding conversational interaction in multiparty conversations: the EVA Corpus

Mlakar

Verdonik

Majhenič

et al. 2022

Lang Resources & Evaluation

View full text Add to dashboard Cite

This paper focuses on gaining new knowledge through observation, qualitative analytics, and cross-modal fusion of rich multi-layered conversational features expressed during multiparty discourse. The outlined research stems from the theory that speech and co-speech gestures originate from the same representation; however, the representation is not solely limited to the speech production process. Thus, the nature of how information is conveyed by synchronously fusing speech and gestures must be investigated in detail. Therefore, this paper introduces an integrated annotation scheme and methodology which opens the opportunity to study verbal (i.e., speech) and non-verbal (i.e., visual cues with a communicative intent) components independently, however, still interconnected over a common timeline. To analyse this interaction between linguistic, paralinguistic, and non-verbal components in multiparty discourse and to help improve natural language generation in embodied conversational agents, a high-quality multimodal corpus, consisting of several annotation layers spanning syntax, POS, dialogue acts, discourse markers, sentiment, emotions, non-verbal behaviour, and gesture units was built and is represented in detail. It is the first of its kind for the Slovenian language. Moreover, detailed case studies show the tendency of metadiscourse to coincide with non-verbal behaviour of non-propositional origin. The case analysis further highlights how the newly created conversational model and the corresponding information-rich consistent corpus can be exploited to deepen the understanding of multiparty discourse.

show abstract

“…Body expressions, and especially self-adaptors, have been shown to be correlated with human affect, depression and psychological distress [13], [14], [15], [16], [17]. Selfadaptors are self-comforting gestures, including any kind of touching on other parts of the body, either dynamically or statically [18], [19].…”

Section: Introductionmentioning

confidence: 99%

Looking at the Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Lin

Orton

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Psychological distress is a significant and growing issue in society. In particular, depression and anxiety are leading causes of disability that often go undetected or late-diagnosed. Automatic detection, assessment, and analysis of behavioural markers of psychological distress can help improve identification and support prevention and early intervention efforts. Compared to modalities such as face, head, and vocal, research investigating the use of the body modality for these tasks is relatively sparse, which is partly due to the limited available datasets and difficulty in automatically extracting useful body features. To enable our research, we have collected and analyzed a new dataset containing full body videos for interviews and self-reported distress labels. We propose a novel approach to automatically detect self-adaptors and fidgeting, a subset of self-adaptors that has been shown to correlate with psychological distress. We perform analysis on statistical body gestures and fidgeting features to explore how distress levels affect behaviors. We then propose a multi-modal approach that combines different feature representations using Multi-modal Deep Denoising Auto-Encoders and Improved Fisher Vector Encoding. We demonstrate that our proposed model, combining audio-visual features with detected fidgeting behavioral cues, can successfully predict depression and anxiety in the dataset.

show abstract

Semantic processing of self-adaptors, emblems, and iconic gestures: An ERP study

Cited by 17 publications

References 35 publications

Influencing choices with conversational primes: How a magic trick unconsciously influences card choices

Influencing choices with conversational primes: How a magic trick unconsciously influences card choices

Understanding conversational interaction in multiparty conversations: the EVA Corpus

Looking at the Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Contact Info

Product

Resources

About