Multi-Task Networks with Universe, Group, and Task Feature Learning

Pentyala, Shiva; Liu, Mengwen; Dreyer, Markus

doi:10.18653/v1/p19-1079

Cited by 17 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by this problem, the joint models [12,13,14] are developed for solving intent detection and slot filling tasks together. Besides, some work [15,5,16,17,18,19], tries to enhance the performance via multi-task learning. These joint models or multi-task learning methods link the two tasks implicitly via applying a joint loss function.…”

Section: Related Workmentioning

confidence: 99%

A Result Based Portable Framework for Spoken Language Understanding

Cheng

Jia

Yang

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Spoken language understanding (SLU), which is a core component of the task-oriented dialogue system, has made substantial progress in the research of single-turn dialogue. However, the performance in multi-turn dialogue is still not satisfactory in the sense that the existing multi-turn SLU methods have low portability and compatibility for other single-turn SLU models. Further, existing multi-turn SLU methods do not exploit the historical predicted results when predicting the current utterance, which wastes helpful information. To gap those shortcomings, in this paper, we propose a novel Resultbased Portable Framework for SLU (RPFSLU). RPFSLU allows most existing single-turn SLU models to obtain the contextual information from multi-turn dialogues and takes full advantage of predicted results in the dialogue history during the current prediction. Experimental results on the public dataset KVRET have shown that all SLU models in baselines acquire enhancement by RPFSLU on multi-turn SLU tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

A Result Based Portable Framework for Spoken Language Understanding

Cheng

Jia

Yang

2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

show abstract

“…In [24], each task has its own encoder and decoder, while all tasks share a representation learning layer and a joint encoding layer. [92] creates encoder modules on different levels, including task level, task group level, and universal level.…”

Section: Modular Architecturesmentioning

confidence: 99%

Multi-Task Learning in Natural Language Processing: An Overview

Chen¹,

Qiang²

2021

Preprint

View full text Add to dashboard Cite

Deep learning approaches have achieved great success in the field of Natural Language Processing (NLP). However, deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks. In recent years, Multi-Task Learning (MTL), which can leverage useful information of related tasks to achieve simultaneous performance improvement on multiple related tasks, has been used to handle these problems. In this paper, we give an overview of the use of MTL in NLP tasks. We first review MTL architectures used in NLP tasks and categorize them into four classes, including the parallel architecture, hierarchical architecture, modular architecture, and generative adversarial architecture. Then we present optimization techniques on loss construction, data sampling, and task scheduling to properly train a multi-task model. After presenting applications of MTL in a variety of NLP tasks, we introduce some benchmark datasets. Finally, we make a conclusion and discuss several possible research directions in this field.

show abstract

“…Meanwhile, some work [10,22] tries to enhance the performance via multi-task learning. These multi-task learning methods link the two tasks implicitly via applying a joint loss function.…”

Section: Related Workmentioning

confidence: 99%

An Effective Non-Autoregressive Model for Spoken Language Understanding

Cheng¹,

Jia

Yang³

2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference latency due to the impatience of humans. Non-autoregressive SLU models clearly increase the inference speed but suffer uncoordinated-slot problems caused by the lack of sequential dependency information among each slot chunk. To gap this shortcoming, in this paper, we propose a novel non-autoregressive SLU model named Layered-Refine Transformer, which contains a Slot Label Generation (SLG) task and a Layered Refine Mechanism (LRM). SLG is defined as generating the next slot label with the token sequence and generated slot labels. With SLG, the non-autoregressive model can efficiently obtain dependency information during training and spend no extra time in inference. LRM predicts the preliminary SLU results from Transformer's middle states and utilizes them to guide the final prediction. Experiments on two public datasets indicate that our model significantly improves SLU performance (1.5% on Overall accuracy) while substantially speed up (more than 10 times) the inference process over the state-of-the-art baseline. CCS CONCEPTS• Computing methodologies → Natural language processing.

show abstract

Multi-Task Networks with Universe, Group, and Task Feature Learning

Cited by 17 publications

References 25 publications

A Result Based Portable Framework for Spoken Language Understanding

A Result Based Portable Framework for Spoken Language Understanding

Multi-Task Learning in Natural Language Processing: An Overview

An Effective Non-Autoregressive Model for Spoken Language Understanding

Contact Info

Product

Resources

About