PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Zhang, Le; Peng, Songyou; Winkler, Stefan

doi:10.1109/taffc.2019.2951656

Cited by 32 publications

(29 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This implies that the Affect category reflects one's personality obviously. Similar conclusions are reported by Depue and Collins (1999) and Zhang et al (2019). In addition, the Function node is the least impactful category node.…”

Section: Modelsupporting

confidence: 89%

Psycholinguistic Tripartite Graph Network for Personality Detection

Yang¹,

Yang²,

Ouyang³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Most of the recent work on personality detection from online posts adopts multifarious deep neural networks to represent the posts and builds predictive models in a data-driven manner, without the exploitation of psycholinguistic knowledge that may unveil the connections between one's language usage and his psychological traits. In this paper, we propose a psycholinguistic knowledge-based tripartite graph network, TrigNet, which consists of a tripartite graph network and a BERT-based graph initializer. The graph network injects structural psycholinguistic knowledge from LIWC, a computerized instrument for psycholinguistic analysis, by constructing a heterogeneous tripartite graph. The graph initializer is employed to provide initial embeddings for the graph nodes. To reduce the computational cost in graph learning, we further propose a novel flow graph attention network (GAT) that only transmits messages between neighboring parties in the tripartite graph. Benefiting from the tripartite graph, TrigNet can aggregate post information from a psychological perspective, which is a novel way of exploiting domain knowledge. Extensive experiments on two datasets show that TrigNet outperforms the existing state-of-art model by 3.47 and 2.10 points in average F1. Moreover, the flow GAT reduces the FLOPS and Memory measures by 38% and 32%, respectively, in comparison to the original GAT in our setting.

show abstract

Section: Modelsupporting

confidence: 89%

Psycholinguistic Tripartite Graph Network for Personality Detection

Yang¹,

Yang²,

Ouyang³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…The unifying idea behind all of the above is deep learning, the utilization of neural networks [15] with many hidden layers, for the purposes of learning complex feature representations from raw data, rather than relying on handcrafted feature extraction. They have shown consistent improvements over their non-deep counterparts across many tasks beyond segmentation [16], [17]. Those approaches usually adopt an "encoder-decoder" structure which could gradually decrease the resolution of the input with the depth of the network in the encoding stage, and then up-sampling and skip connections are applied to recover the resolution of the input in the decoding stage.…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Self-Taught Deep Learning for Finger Bones Segmentation

Zhao¹,

Zhang²,

Chen³

et al. 2019

2019 IEEE EMBS International Conference on Biomedical &Amp; Health Informatics (BHI)

Self Cite

View full text Add to dashboard Cite

Segmentation stands at the forefront of many highlevel vision tasks. In this study, we focus on segmenting finger bones within a newly introduced semi-supervised self-taught deep learning framework which consists of a student network and a stand-alone teacher module. The whole system is boosted in a life-long learning manner wherein each step the teacher module provides a refinement for the student network to learn with newly unlabeled data. Experimental results demonstrate the superiority of the proposed method over conventional supervised deep learning methods.

show abstract

“…As there is converging evidence demonstrating that nonverbal behaviours are significant predictors of personality, most existing automatic approaches attempt to predict personality traits from nonverbal audio-visual behaviours (e.g., facial expressions, vocal prosody, etc.) [10,17,29,49,53]. Majority of these works focus on analysing an individual's observable behaviours, disregarding the interpersonal interaction context and cues (e.g., interpersonal behaviours due to dyadic / triadic /group interactions).…”

Section: Introductionmentioning

confidence: 99%

Personality Recognition by Modelling Person-specific Cognitive Processes using Graph Representation

Shao

Song

Jaiswal

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Recent research shows that in dyadic and group interactions individuals' nonverbal behaviours are influenced by the behaviours of their conversational partner(s). Therefore, in this work we hypothesise that during a dyadic interaction, the target subject's facial reactions are driven by two main factors: (i) their internal (personspecific) cognition, and (ii) the externalised nonverbal behaviours of their conversational partner. Subsequently, our novel proposition is to simulate and represent the target subject's (i.e., the listener) cognitive process in the form of a person-specific CNN architecture whose input is the audio-visual non-verbal cues displayed by the conversational partner (i.e., the speaker), and the output is the target subject's (i.e., the listener) facial reactions. We then undertake a search for the optimal CNN architecture whose results are used to create a person-specific graph representation for recognising the target subject's personality. The graph representation, fortified with a novel end-to-end edge feature learning strategy, helps with retaining both the unique parameters of the person-specific CNN and the geometrical relationship between its layers. Consequently, the proposed approach is the first work that aims to recognize the true (self-reported) personality of a target subject (i.e., the listener) from the learned simulation of their cognitive process (i.e., parameters of the person-specific CNN). The experimental results show that the CNN architectures are well associated with target subjects' personality traits and the proposed approach clearly outperforms multiple existing approaches that predict personality directly from non-verbal behaviours. In light of these findings, this work opens up a new avenue of research for predicting and recognizing socioemotional phenomena (personality, affect, engagement etc.) from simulations of person-specific cognitive processes.

show abstract

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Cited by 32 publications

References 56 publications

Psycholinguistic Tripartite Graph Network for Personality Detection

Psycholinguistic Tripartite Graph Network for Personality Detection

Semi-Supervised Self-Taught Deep Learning for Finger Bones Segmentation

Personality Recognition by Modelling Person-specific Cognitive Processes using Graph Representation

Contact Info

Product

Resources

About