Controlling Personality Style in Dialogue with Zero-Shot Prompt-Based Learning

Abstract

Prompt-based or in-context learning has been shown to achieve high zeroshot performance on many natural language generation (NLG) tasks. Here we explore the performance of prompt-based learning for simultaneously controlling the personality and the semantic accuracy of an NLG for task-oriented dialogue. We experiment with prompt-based learning on the PERSONAGE restaurant recommendation corpus to generate semantically and stylistically-controlled text for 5 different Big-5 personality types: agreeable, disagreeable, conscientious, unconscientious, and extravert. We test two different classes of discrete prompts to generate utterances for a particular personality style: (1) prompts that demonstrate generating directly from a meaning representation that includes a personality specification; and (2) prompts that rely on first converting the meaning representation to a textual pseudo-reference, and then using the pseudo-reference in a textual style transfer (TST) prompt. In each case, we show that we can vastly improve performance by over-generating outputs and ranking them, testing several ranking functions based on automatic metrics for semantic accuracy, personality-match, and fluency. We also test the effect of providing examples of multiple personalities, and of different sampling strategies and numbers of examples, as well as testing whether NLG personality demonstrations from the restaurant domain can be used with meaning representations for the video game domain to generate personality stylized utterances about video games. Our findings show that the TST prompts produces the highest semantic accuracy (78.46% for restaurants and 87.6% for video games) and personality accuracy (100% for restaurants and 97% for video games). Our results on transferring personality style to video game utterances are surprisingly good. To our knowledge, there is no previous work testing the application of prompt-based learning to simultaneously controlling both style and semantic accuracy in NLG.

Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking

“…For example, curated opinionated utterances from user reviews that had been marked with exclamation points, and then used the exclamation points as a way to control the production of exaggerated opinions. Reed et al (2020) used token supervision to control the production of recommendation as opposed to inform dialogue acts where recommendation DAs stated that a particular restaurant was the best and then justified the recommendation with attributes from the MR. Ramirez et al (2023) used PBL with similar prompts to control the expression of Big 5 personality types , rather than dialogue acts.…”