Auxiliary Capsules for Natural Language Understanding

A survey of joint intent detection and slot-filling models in natural language understanding

“…The method of using SLU as fine-tuning with pre-training on another task, or vice versa, has shown improvements in the SLU performance. However the results of [Staliūnaitė and Iacobacci 2020], echoing those of [Louvan and Magnini 2019] on slot tagging, indicate a parsimonious approach to adding extra tasks simultaneously more often yields a better result.…”

“…This explicit and direct feedback is stronger than the implicit or indirect joint learning typically found in RNN models. [Staliūnaitė and Iacobacci 2020] extended this work to a multi-task setting with extra mid-level capsules for NER and POS labels, with mixed results. [Wen et al 2018] Using hierarchy and context Two layer (Bi)LSTM [Wang et al 2018c] Capturing local semantic information CNN, BiLSTM encoder decoder [Firdaus et al 2018a] Domain dependence Ensemble model, GRU Slow training time Progressive multi-task model using user information [Li et al 2018a] Correlation of different tasks Multi-task model incl.…”

“…CRF and self attention over BERT [Ni et al 2020] Model transfer, IoT Pipeline structure from medical analogue [Krone et al 2020] Unseen labels Few-shot meta-learning [Bhathiya and Thayasivam 2020] Unseen labels, language transfer Few-shot meta-learning [Tang et al 2020] Linear chain CRF limitations GCN based CRF [Staliūnaitė and Iacobacci 2020] Extend capsule network Capsule network with MTL [Han et al 2020] Explicit interaction, word-label information Bidirection, attention, GCN graph 5.2.6 Bi-directional models. A model where there is a pipeline from one sub-task to the other may be seen as unidirectional.…”

“…[ Staliūnaitė and Iacobacci 2020] incorporated POS and NER tagging simultaneously with slot tagging and intent detection using a capsule network, however the results were generally poorer when both NER and POS were included rather than just one, and mixed for different data sets indicating a generalisability issue.…”

State of the Art in Intent Detection and Slot Filling for Question Answering System: A Systematic Literature Review

“…Furthermore, many previous research articles have been published and have discussed the researcher's implementation on the topic of ID and IC with slot filling. Papers [1,8,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37] have discussed and presented their approach or implementation on developing ID and IC with slot filling using various frameworks, methodologies, and techniques. Various frameworks, methodologies, approaches, techniques and algorithms have been implemented by these researchers, whereas for instance, [19,22,24,26,27,28,31,34,35,36] have implemented techniques of Bidirectional Long Short-Term Memory ID (BiLSTM) with Conditional Random Forest (CRF) in their implementation of ID and IC with slot filling.…”