A Corpus-Based Approach for Cooperative Response Generation in a Dialog System

Wu, Zhiyong; Meng, Helen; Ning, Hui; Tse, Sam C.

doi:10.1007/11939993_63

Cited by 4 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After the dialog model issues a proper action as the response, the natural language generator is responsible for translating the representation of the response semantics into text, which is then passed to the Text-to-Speech (TTS) synthesizer to generate audio output. Advances in speech and language technologies have made SDSs an important research area and have brought about systems in a wide variety of application domains, such as bus schedule inquiries [1], flight information [2], stock market information delivery [3], tourist guides [4] and student tutoring [5]. As SDSs are becoming increasingly pervasive, their ultimate goal is to satisfy the users' needs with good performance yielding a good user experience.…”

Section: Introductionmentioning

confidence: 99%

Predicting User Satisfaction in Spoken Dialog System Evaluation With Collaborative Filtering

Yang

Levow

Meng

2012

IEEE J. Sel. Top. Signal Process.

Self Cite

View full text Add to dashboard Cite

Abstract-We propose a collaborative filtering (CF) model to predict user satisfaction in SDS evaluation. Inspired by the use of CF in recommendation systems, where a user's preference for a new item is assume to resemble that for similar items rated previously, we adapt the idea to predict user evaluations of unrated dialogs based on the ratings received by similar dialogs. Ratings of dialogs are gathered by crowdsourcing through Amazon Mechanical Turk. A reference baseline is provided by a linear regression model (LRM) based on the PARADISE framework. We present two versions of the CF model. First, the item-based collaborative filtering model (ICFM) clusters rated dialogs and builds an LRM for each cluster. The rating of an unseen dialog is predicted by the LRM of its most similar cluster. Second, the extended ICFM (EICFM) separates dialog features into user-related and system-related groups, to build LRMs for these separately. Experimental results on dialogs from the Let's Go! system show both ICFM and EICFM can significantly improve the proportion of variability explained by the LRM. We also demonstrate the generalizability of the CF model to a new dialog corpus from the systems in the Spoken Dialog Challenge (SDC) 2010.

show abstract

Section: Introductionmentioning

confidence: 99%

Predicting User Satisfaction in Spoken Dialog System Evaluation With Collaborative Filtering

Yang

Levow

Meng

2012

IEEE J. Sel. Top. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

Crowdsourcing for Spoken Dialog System Evaluation

Yang

Levow

Meng

2013

Crowdsourcing for Speech Processing

View full text Add to dashboard Cite

Cross-Modality Semantic Integration With Hypothesis Rescoring for Robust Interpretation of Multimodal User Interactions

Hui

Meng

2009

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-We develop a framework pertaining to automatic semantic interpretation of multimodal user interactions using speech and pen gestures. The two input modalities abstract the user's intended message differently into input events, e.g., key terms/phrases in speech or different types of gestures in the pen modality. The proposed framework begins by generating partial interpretations for each input event as a ranked list of hypothesized semantics. We devise a cross-modality semantic integration procedure to align the pair of hypothesis lists between every speech input event and every pen input event in a multimodal expression. This is achieved by the Viterbi alignment algorithm that enforces the temporal ordering of the input events as well as the semantic compatibility of aligned events. The alignment enables generation of a unimodal, verbalized paraphrase that is semantically equivalent to the original multimodal expression. Our experiments are based on a multimodal corpus in the domain of city navigation. Application of the cross-modality integration procedure to near-perfect (manual) transcripts of the speech and pen modalities show that correct unimodal paraphrases are generated for over 97% of the training and test sets. However, if we replace with automatic speech and pen recognition transcripts, the performance drops to 53.7% and 54.8% for the training and test sets, respectively. In order to address this issue, we devised the hypothesis rescoring procedure that evaluates all candidates of cross-modality integration derived from multiple recognition hypotheses from each modality. The rescoring function incorporates the integration score, -best purity of recognized spoken locative expressions, as well as distances between coordinates of recognized pen gestures and their interpreted icons on the map. Application of cross-modality hypothesis rescoring improved the performance to 67.5% and 69.9% for the training and test sets, respectively. Index Terms-Joint integration, human-computer interaction, hypothesis rescoring, multimodal input, pen gesture, perplexity, robust interpretation, spoken input.

show abstract

A Corpus-Based Approach for Cooperative Response Generation in a Dialog System

Cited by 4 publications

References 7 publications

Predicting User Satisfaction in Spoken Dialog System Evaluation With Collaborative Filtering

Predicting User Satisfaction in Spoken Dialog System Evaluation With Collaborative Filtering

Crowdsourcing for Spoken Dialog System Evaluation

Cross-Modality Semantic Integration With Hypothesis Rescoring for Robust Interpretation of Multimodal User Interactions

Contact Info

Product

Resources

About