Asgard: A portable architecture for multilingual dialogue systems

Abstract

Spoken dialogue systems have been studied for years, yet portability is still one of the biggest challenges in terms of language extensibility, domain scalability, and platform compatibility. In this work, we investigate the portability issue from the language understanding perspective and present the Asgard architecture, a CRF-based (Conditional Random Fields) and crowd-sourcing-centered framework, which supports expert-free development of multilingual dialogue systems and seamless deployment to mobile platforms. Combinations of linguistic and statistical features are employed for multilingual semantic understanding, such as n-grams, tokenization and part-of-speech. English and Mandarin systems in various domains (movie, flight and restaurant) are implemented with the proposed framework and ported to mobile platforms as well, which sheds lights on large-scale speech App development.

Spoken Content Retrieval—Beyond Cascading Speech Recognition with Text Retrieval

“…Such dialogue processes have been well studied for other tasks such as air ticket booking, city guides, and so on [352], [354], [355], so extending experiences in those tasks to interactive retrieval is natural, for example, considering a statistical model such as a Markov Decision Process (MDP) [356]. In MDP, the actions taken by the system is chosen based on the states, which can be one or more continuous or quantized values (here the estimated quality of the present retrieved results based on all the input entered so far by the user (U1,U2,U3,U4)).…”

Query understanding enhanced by hierarchical parsing structures

“…The class labels include Award, Title, Opinion, Year, Origin, Genre, Director, Plot, Quote, Actor, Soundtrack, Character, and Other. Different from the labeling HITs deployed in the previous study [20] where most segments were short phrases, in this task workers tend to select long constituents as segments, such as a movie's origin, viewers' opinions, and plot descriptions. We randomly divided the annotated queries into training (80%) and test (20%) sets.…”

“…And the combination of dependency features achieved an F1-score of 86.40%, outperforming the baseline of 85.32%. To evaluate the proposed approach in another domain, we also conducted the experiments on a set of 9,991 restaurant queries, which was collected in a previous study [20] and contained complex query constituents (e.g., Amenity, Opinion). The queries were randomly divided into training (80%) and test (20%) sets as well.…”

“…To collect natural language queries from real users, we employed the annotation toolkit previously developed in [20], which uses Amazon Mechanical Turk (AMT) [21] as the crowd-sourcing platform. In order to collect semantically complex queries, we designed the query generation HIT (Human Intelligent Task) as a verbal Charade game: in each HIT, the worker was asked to make up a query for searching a selected movie, without mentioning the movie title (as demonstrated in Figure 4).…”

“…Some studies have also employed lexical features with external dictionaries and shallow parsing features such as Part-of-Speech (POS) tags [1] [20], which have proved to work well on short phrase-based segments (e.g., named entities). However, for semantically complex queries like the example in Figure 1, lexicons or shallow parsing features are not sufficient for capturing the sentencelevel long segments (e.g., "<Plot>").…”