Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations

Malhotra, Ganeshan; Waheed, Abdul; Srivastava, Aseem; Akhtar, Md. Shad; Chakraborty, Tanmoy

doi:10.1145/3488560.3498509

Cited by 17 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly, English conversations by human therapists display also higher levels of trust, joy and anticipation (up to 60%) during the middle of conversations and at their end. These patterns reflect a practice where communicative intentions steer towards providing support with trust and anticipation into the future before the end of the conversation itself [8]. Analogously, ChatGPT therapists produce conversations progressively richer and richer in anticipation, joy and anxiety within the final rounds of conversation quips.…”

Section: Preprintmentioning

confidence: 90%

See 1 more Smart Citation

Introducing CounseLLMe: A dataset of simulated mental health dialogues for comparing LLMs like Haiku, LLaMAntino and ChatGPT against humans

De Duro,

Improta,

Stella

2024

Preprint

View full text Add to dashboard Cite

We introduce CounseLLMe as a multilingual, multimodal dataset of 400 simulated mental health counselling dialogues between two state-of-the-art Large Language Models (LLMs). These conversations - of 20 quips each - were generated either in English (using OpenAI's GPT 3.5 and Claude-3's Haiku) or Italian (with Claude-3's Haiku and LLaMAntino) and with prompts tuned also with the help of a professional in psychotherapy. We investigate the resulting conversations through comparison against human mental health conversations on the same topic of depression. To compare linguistic features, knowledge structure and emotional content between LLMs and humans, we employed textual forma mentis networks, i.e. cognitive networks where nodes represent concepts and links indicate syntactic or semantic relationships between concepts in the dialogues' quips. We find that the emotional structure of LLM-LLM English conversations matches the one of humans in terms of patient-therapist trust exchanges, i.e. 1 in 5 LLM-LLM quips contain trust along 10 conversational turns versus the $24\%$ rate found in humans. ChatGPT and Haiku's simulated English patients can also reproduce human feelings of conflict and pessimism. However, human patients display non-negligible levels of anger/frustration that is missing in LLMs. Italian LLMs' conversations are worse in reproducing human patterns. All LLM-LLM conversations reproduced human syntactic patterns of increased absolutist pronoun usage in patients and second-person, trust-inducing, pronoun usage in therapists. Our results indicate that LLMs can realistically reproduce several aspects of human patient-therapist conversations and we thusly release CounseLLMe as a public dataset for novel data-informed opportunities in mental health and machine psychology.

show abstract

Section: Preprintmentioning

confidence: 90%

“…To enable human-level comparison, we analyzed the outputs of the LLMs alongside the ones of real human responses from the HOPE dataset [8]. This dataset contains transcripts of almost 13k conversations between psychotherapists and patients, extracted from publicly available YouTube videos.…”

Section: Comparison Of Counsellme With the Hope Datasetmentioning

confidence: 99%

Introducing CounseLLMe: A dataset of simulated mental health dialogues for comparing LLMs like Haiku, LLaMAntino and ChatGPT against humans

De Duro,

Improta,

Stella

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Duran et al [14].during the encoding phase, integrate act and sentence encoding using BERT, systematically comparing and validating various encoding mechanisms' performance differences in act classification. Malhotra et al [15]. leverage gated recurrent units and a time sliding window to perceive local and global contextual cues within dialogues for act recognition.…”

Section: Based On Sequence Labelingmentioning

confidence: 99%

A Personalized Model Integrating Personality for Joint Dialogue Emotion Classification and Act Recognition

Ma,

Zhou,

et al. 2024

Preprint

View full text Add to dashboard Cite

The joint dialogue emotion classification and act recognition tasks essentially involves the simultaneous classification of emotional and act labels for each utterance in a dialogue. Existing dual-tasks models can recognize emotion and act based on utterance context but overlook the interaction between discourse under the influence of personality and the correlation between dual-tasks. Utter- ances within the same personality and across different personalities exhibit explicit temporal dependencies in the past and future. However, existing models fail to explicitly address the temporal relationships in the interaction between personality and dual-tasks, resulting in a reasoning gap in the temporal corre- lation of interactions under long-term personality. In this paper, A Personalized Model Integrating Personality for Joint Dialogue Emotion Classification and Act RecognitionPEDR-GATis proposed to solve these problems.Firstly, we uti- lize personality information and label distributions to provide initial estimates in semantics, strengthening the association between local semantics guided by personality and task labels. Secondly, the model employs graph attention mech- anisms to extract global semantic temporal information and causal temporal relationships between the two tasks. Finally, PEDR-GAT, by acquiring initial local label information guided by personality and integrating personality-guided global semantic label information, introduces a common attention mechanism to maintain long-term dependencies and supplement short-term information.The experiments were conducted on three benchmark datasets: Mastodon social network (Mastodon), A Manually Labelled Multi-turn Dialogue Dataset (Dai- lyDialog), and A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI (CPED). Compared to the baseline models, there were improvements in F1 scores for emotion classification and act recognition by 1.9% and 0.7% (Mastodon),3.3% and 1.3% (DailyDialog), and 5.3% and 2.3%(CPED), respectively. The results indicate that personality-guided semantic label acquisition enhances the association between semantics and labels, and the graph attention mechanism with personality fusion reasoning can stimulate the cognitive system’s understanding of potential causal information in abstraction.

show abstract

“…Computational approaches using natural language processing offer the potential to move past human limits of attention and reproducibility 19,[27][28][29][30][31][32] . Improvements in computational power, the growing ease of recording and transcribing therapy sessions, and a shift to computer-mediated communication in healthcare (i.e., telehealth) make this feasible 19,22,33,34 . Supervised machine learning has provided insight into important constructs such as empathy and therapeutic interventions but rely on timeconsuming and sometimes inconsistent human evaluation, making inspectability and reproducibility a challenge 26,35,36 .…”

Section: Introductionmentioning

confidence: 99%

A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency

Miner

Fleming

Haque

et al. 2022

npj Mental Health Res

View full text Add to dashboard Cite

Although individual psychotherapy is generally effective for a range of mental health conditions, little is known about the moment-to-moment language use of effective therapists. Increased access to computational power, coupled with a rise in computer-mediated communication (telehealth), makes feasible the large-scale analyses of language use during psychotherapy. Transparent methodological approaches are lacking, however. Here we present novel methods to increase the efficiency of efforts to examine language use in psychotherapy. We evaluate three important aspects of therapist language use - timing, responsiveness, and consistency - across five clinically relevant language domains: pronouns, time orientation, emotional polarity, therapist tactics, and paralinguistic style. We find therapist language is dynamic within sessions, responds to patient language, and relates to patient symptom diagnosis but not symptom severity. Our results demonstrate that analyzing therapist language at scale is feasible and may help answer longstanding questions about specific behaviors of effective therapists.

show abstract

Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations

Cited by 17 publications

References 22 publications

Introducing CounseLLMe: A dataset of simulated mental health dialogues for comparing LLMs like Haiku, LLaMAntino and ChatGPT against humans

Introducing CounseLLMe: A dataset of simulated mental health dialogues for comparing LLMs like Haiku, LLaMAntino and ChatGPT against humans

A Personalized Model Integrating Personality for Joint Dialogue Emotion Classification and Act Recognition

A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency

Contact Info

Product

Resources

About