A Result Based Portable Framework for Spoken Language Understanding

Cheng, Lizhi; Jia, Weijia; Yang, Wenmian

doi:10.1109/icme51207.2021.9428384

Cited by 9 publications

(25 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Actually, the historical utterance with higher semantic similarity to the current dialogue should have a more signiicant inluence [25]. Moreover, Cheng et al [9] proved that the predicted results in dialogue history also contain important semantics and are even more speciic than utterances in SLU tasks. Therefore, both historical utterances and historical results should be considered to obtain salient information from dialogue history.…”

Section: Salient History Atentionmentioning

confidence: 99%

“…Qin et al [37] then concern about the impact of SF results in their model but still sufering the long inference latency. Inspired by [50], Cheng et al [9] propose a portable framework RPFSLU, which contains a two-round predicting period. RPFSLU utilizes the semantic information of the irst round prediction results to guide the second round prediction by a represent learning process.…”

Section: Single-turn Slumentioning

confidence: 99%

“…Cheng et al [9] then propose RPFSLU, which attain contextual information from both historical utterance and historical predicted results. Meanwhile, RPFSLU is a portable framework that can easily let nearly all RNN-based single-turn SLU models also be available in multi-turn SLU tasks.…”

Section: Multi-turn Slumentioning

confidence: 99%

“…The query (Q), key (K), and value (V) of our history-attention is generated by the current utterance, historical utterances, and historical results, respectively. Since the predicted results of SLU in dialogue history contain essential and speciic semantics (proved byCheng et al [9]), our SHA module could obtain the hidden states of dialogue history with salient historical information. Subsequently, we utilize the hidden states of dialogue history to guide the prediction of the current-turn utterance.…”

Section: Introductionmentioning

confidence: 97%

“…Some studies [2,3,7] obtain historical information via some lengthy and complicated architectures and thus consume a lot of inference time. Cheng et al [9] propose RPFSLU framework which employ an RNN-based portable plugin to obtain historical information for state-of-art single-turn SLU models [16,35]. Unfortunately, RPFSLU still needs to calculate the hidden states of each historical utterance in succession, which is low eiciency.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Capture Salient Historical Information: A Fast and Accurate Non-autoregressive Model for Multi-turn Spoken Language Understanding

Cheng¹,

jia²,

Yang³

2022

ACM Trans. Inf. Syst.

Self Cite

View full text Add to dashboard Cite

Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference facing the impatience of human users. Existing work increases inference speed by designing non-autoregressive models for single-turn SLU tasks but fails to apply to multi-turn SLU in confronting the dialogue history. The intuitive idea is to concatenate all historical utterances and utilize the non-autoregressive models directly. However, this approach seriously misses the salient historical information and suffers from the uncoordinated-slot problems. To overcome those shortcomings, we propose a novel model for multi-turn SLU named Salient History Attention with Layer-Refined Transformer (SHA-LRT), which composes of an SHA module, a Layer-Refined Mechanism (LRM), and a Slot Label Generation (SLG) task. SHA captures salient historical information for the current dialogue from both historical utterances and results via a well-designed history-attention mechanism. LRM predicts preliminary SLU results from Transformer’s middle states and utilizes them to guide the final prediction, and SLG obtains the sequential dependency information for the non-autoregressive encoder. Experiments on public datasets indicate that our model significantly improves multi-turn SLU performance (17.5% on Overall) with accelerating (nearly 15 times) the inference process over the state-of-the-art baseline as well as effective on the single-turn SLU tasks.

show abstract

Section: Salient History Atentionmentioning

confidence: 99%

Section: Single-turn Slumentioning

confidence: 99%

Section: Multi-turn Slumentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Capture Salient Historical Information: A Fast and Accurate Non-autoregressive Model for Multi-turn Spoken Language Understanding

Cheng¹,

jia²,

Yang³

2022

ACM Trans. Inf. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

An Effective Non-Autoregressive Model for Spoken Language Understanding

Cheng¹,

Jia

Yang³

2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

Self Cite

View full text Add to dashboard Cite

Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference latency due to the impatience of humans. Non-autoregressive SLU models clearly increase the inference speed but suffer uncoordinated-slot problems caused by the lack of sequential dependency information among each slot chunk. To gap this shortcoming, in this paper, we propose a novel non-autoregressive SLU model named Layered-Refine Transformer, which contains a Slot Label Generation (SLG) task and a Layered Refine Mechanism (LRM). SLG is defined as generating the next slot label with the token sequence and generated slot labels. With SLG, the non-autoregressive model can efficiently obtain dependency information during training and spend no extra time in inference. LRM predicts the preliminary SLU results from Transformer's middle states and utilizes them to guide the final prediction. Experiments on two public datasets indicate that our model significantly improves SLU performance (1.5% on Overall accuracy) while substantially speed up (more than 10 times) the inference process over the state-of-the-art baseline. CCS CONCEPTS• Computing methodologies → Natural language processing.

show abstract

A Survey of Joint Intent Detection and Slot Filling Models in Natural Language Understanding

et al. 2022

View full text Add to dashboard Cite

Intent classification, to identify the speaker’s intention, and slot filling, to label each token with a semantic type, are critical tasks in natural language understanding. Traditionally the two tasks have been addressed independently. More recently joint models, that address the two tasks together, have achieved state-of-the-art performance for each task, and have shown there exists a strong relationship between the two. In this survey we bring the coverage of methods up to 2021 including the many applications of deep learning in the field. As well as a technological survey we look at issues addressed in the joint task, and the approaches designed to address these issues. We cover data sets, evaluation metrics, experiment design and supply a summary of reported performance on the standard data sets.

show abstract

A Result Based Portable Framework for Spoken Language Understanding

Cited by 9 publications

References 20 publications

Capture Salient Historical Information: A Fast and Accurate Non-autoregressive Model for Multi-turn Spoken Language Understanding

Capture Salient Historical Information: A Fast and Accurate Non-autoregressive Model for Multi-turn Spoken Language Understanding

An Effective Non-Autoregressive Model for Spoken Language Understanding

A Survey of Joint Intent Detection and Slot Filling Models in Natural Language Understanding

Contact Info

Product

Resources

About