Modeling Empathic Similarity in Personal Narratives

Abstract

The most meaningful connections between people are often fostered through expression of shared vulnerability and emotional experiences in personal narratives. We introduce a new task of identifying similarity in personal stories based on empathic resonance, i.e., the extent to which two people empathize with each others' experiences, as opposed to raw semantic or lexical similarity, as has predominantly been studied in NLP. Using insights from social psychology, we craft a framework that operationalizes empathic similarity in terms of three key features of stories: main events, emotional trajectories, and overall morals or takeaways. We create EM-PATHICSTORIES, a dataset of 1,500 personal stories annotated with our empathic similarity features, and 2,000 pairs of stories annotated with empathic similarity scores. Using our dataset, we finetune a model to compute empathic similarity of story pairs, and show that this outperforms semantic similarity models on automated correlation and retrieval metrics. Through a user study with 150 participants, we also assess the effect our model has on retrieving stories that users empathize with, compared to naive semantic similarity-based retrieval, and find that participants empathized significantly more with stories retrieved by our model. Our work has strong implications for the use of empathy-aware models to foster human connection and empathy between people.

Empathy Towards AI vs Human Experiences: The Role of Transparency in Mental Health and Social Support Chatbot Design (Preprint)

“…The retrieved stories are matched based on similarity of the embeddings of stories, and generated stories are generated on-the-fly given the user's story as a prompt. We used ChatGPT to generate a set of 1,568 stories using seed stories from the EMPATHICSTORIES dataset [24]. Stories generated by ChatGPT were prompted with a context story and the following instruction: Write a story from your own life that the narrator would empathize with.…”

“…While many methods exist to retrieve semantically similar pieces of text [25], few focus on retrieving stories that users would emotionally resonate with given their own story context. As such, we use a fine-tuned BART-base model from Shen et al, which is trained on the EMPATHICSTORIES dataset, a corpus containing pairs of stories each annotated with an "empathic similarity" score from 1-4, where empathic similarity refers to how likely the narrators of both stories would empathize with one another [24]. Using this model, we improved retrieval of stories that are empathetically relevant to a user's own personal story.…”

“…As such, mental health or social support chatbots powered by AI represent a population sourced from large quantities of human data, but still fall short of human-written stories in their empathic quality [9], [21], [34], [35]. This appropriation of human experiences could be subverted by using AI to instead, retrieve more empathically similar texts between human authors [24], such as in online social support group settings, or to mediate human-human communications, such as between patient and therapist [36].…”