Where do thewords come from? Learning models for word choice and ordering from spoken dialog corpora

Abstract

Most existing generation systems for spoken dialog require the system engineer to specify by hand the words to be used in system prompts. However, the existence of corpora of spoken dialog makes it possible to acquire the words and structure of system prompts automatically. In this paper, we construct statistical models for generating system prompts, both for word choice and for word ordering. We evaluate these models using a human-computer dialog multicorpus and a human-human dialog corpus. Our results show that statistical models for word choice can work well, while more work is needed on statistical models for word ordering.

Context‐Sensitive Natural Language Generation: From Knowledge‐Driven to Data‐Driven Techniques

“…Trainable approaches to generation such as supervised learning, example‐based learning (DeVault et al ; Stent et al ) or n‐gram models perform best when trained from a large number of well‐balanced and representative training examples. This can make them faster to develop than knowledge‐driven approaches (given that training data are available) and can replace human intuition in areas where human analysis is costly or the data are diverse and difficult to analyse.…”