Converting the Point of View of Messages Spoken to Virtual Assistants

Abstract

Virtual Assistants can be quite literal at times. If a user says tell Bob I love him, most virtual assistants will extract the message I love him and send it to the user's contact named Bob, rather than properly converting the message to I love you. We designed a system that takes a voice message from one user, converts the point of view of the message, and then delivers the result to its target user. We developed a rulebased model, which integrates a linear text classification model, part-of-speech tagging, and constituency parsing with rule-based transformation methods. We also investigated Neural Machine Translation (NMT) approaches, including traditional recurrent networks, Copy-Net, and T5. We explored 5 metrics to gauge both naturalness and faithfulness automatically, and we chose to use BLEU plus METEOR for faithfulness, as well as relative perplexity using a separately trained language model (GPT) for naturalness. Transformer-Copynet and T5 performed similarly on faithfulness metrics, with T5 scoring 63.8 for BLEU and 83.0 for ME-TEOR. CopyNet was the most natural, with a relative perplexity of 1.59. CopyNet also has 37 times fewer parameters than T5. We have publicly released our dataset, which is composed of 46,565 crowd-sourced samples.

Unsupervised Abstractive Dialogue Summarization with Word Graphs and POV Conversion

“…In fact, the English grammar is mostly descriptive rather than prescriptive -no set of official rules dictated by a single governing authority exists. Even so, rule based POV conversion does provide a strong baseline compared to state-of-the-art techniques, such as end-to-end Transformer networks (Lee et al, 2020). In this study, we limit our scope to rule-based conversion because only the rule-based system among all tested methods in Lee et al (2020) confers to the unsupervised nature of this paper.…”

“…Even so, rule based POV conversion does provide a strong baseline compared to state-of-the-art techniques, such as end-to-end Transformer networks (Lee et al, 2020). In this study, we limit our scope to rule-based conversion because only the rule-based system among all tested methods in Lee et al (2020) confers to the unsupervised nature of this paper. We encourage further research into integrating more advanced reported speech conversion techniques into the abstractive summarization pipeline.…”

“…We notably omit prepend rules suggested in (Lee et al, 2020), because the input domain of our summarization system is unbounded, unlike with taskoriented spoken commands for virtual assistants. We also leave tense conversion for future research.…”

“…While our methods demonstrate improved benchmark results, its mostly heuristic nature leaves much room for enhancement through integration of statistical models. POV conversion in particular can benefit from deep learning-based approaches (Lee et al, 2020). With recent advances in unsupervised sequence to sequence transduction (Li et al, 2020;He et al, 2020), we expect further research into more advanced POV conversion techniques will improve unsupervised dialogue summarization.…”