Memory Augmented Attention Model for Chinese Implicit Discourse Relation Recognition

Liu, Yang; Zhang, Jiajun; Zong, Chengqing

doi:10.1007/978-3-319-69005-6_34

Cited by 5 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fan et al [21] proposed a BiLSTM-based model combining self-attention mechanism and syntactic information. Liu and Li [83] Some other neural models also adopt attention mechanisms for the IDRR task [3,28,29,85,113,115]. For example, Bai and Zhao [3] proposed to learn different granularity levels of representations via an attention-augment neural model.…”

Section: Attention Mechanismmentioning

confidence: 99%

A Survey of Implicit Discourse Relation Recognition

Wang¹,

Wang²

2022

Preprint

View full text Add to dashboard Cite

A discourse containing one or more sentences describes daily issues and events for people to communicate their thoughts and opinions. As sentences are normally consist of multiple text segments, correct understanding of the theme of a discourse should take into consideration of the relations in between text segments. Although sometimes a connective exists in raw texts for conveying relations, it is more often the cases that no connective exists in between two text segments but some implicit relation does exist in between them. The task of implicit discourse relation recognition (IDRR) is to detect implicit relation and classify its sense between two text segments without a connective. Indeed, the IDRR task is important to diverse downstream natural language processing tasks, such as text summarization, machine translation and so on. This article provides a comprehensive and up-to-date survey for the IDRR task. We first summarize the task definition and data sources widely used in the field. We categorize the main solution approaches for the IDRR task from the viewpoint of its development history. In each solution category, we present and analyze the most representative methods, including their origins, ideas, strengths and weaknesses. We also present performance comparisons for those solutions experimented on a public corpus with standard data processing procedures. Finally, we discuss future research directions for discourse relation analysis.

show abstract

Section: Attention Mechanismmentioning

confidence: 99%

A Survey of Implicit Discourse Relation Recognition

Wang¹,

Wang²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Rönnqvist et al (2017) proposed a Bi-LSTM model with attention mechanism to link two arguments by inserting special labels. Liu et al (2017) provided a memory augmented attention model that used memory slots to store the interactions between two input arguments.…”

Section: Related Workmentioning

confidence: 99%

“…In the literature, most of previous studies focused on English, with only a few on Chinese. Compared with traditional feature-based methods (Pitler et al, 2009;Lin et al, 2009;Kong and Zhou, 2017) that directly rely on feature engineering, recent neural network models (Liu et al, 2017;Qin et al, 2017;Guo et al, 2018;Bai and Zhao, 2018) can capture deeper semantic cues and learn better representations (Zhang et al, 2015). In particular, most neural network-based methods encode arguments using variants of Bi-LSTM or CNN (Qin et al, 2016;Guo et al, 2018) and propose various models (e.g., the gated relevance network, the encoder-decoder model, and interactive attention) to measure the semantic relevance (Chen et al, 2016;Cianflone and Kosseim, 2018;Guo et al, 2018) Due to the large differences between the hypotactic English language and the paratactic Chinese language, English-based models, which rely heavily on sentence-level representations, may not function well on Chinese.…”

Section: Introductionmentioning

confidence: 99%

Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

Kong

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

In the literature, most of the previous studies on English implicit discourse relation recognition only use sentence-level representations, which cannot provide enough semantic information in Chinese due to its unique paratactic characteristics. In this paper, we propose a topic tensor network to recognize Chinese implicit discourse relations with both sentencelevel and topic-level representations. In particular, besides encoding arguments (discourse units) using a gated convolutional network to obtain sentence-level representations, we train a simplified topic model to infer the latent topic-level representations. Moreover, we feed the two pairs of representations to two factored tensor networks, respectively, to capture both the sentence-level interactions and topiclevel relevance using multi-slice tensors. Experimentation on CDTB, a Chinese discourse corpus, shows that our proposed model significantly outperforms several state-of-the-art baselines in both micro and macro F1-scores.

show abstract

“…As mentioned above, various methods have been proposed for English (PDTB), while the Chinese task has received little attention in the literature. Liu et al [47] utilized an attention-based neural network to represent arguments and employed an external memory network to preserve crucial information for Chinese task. We argue that Chinese needs word segmentation and other pre-processing, which is more sensitive to multi-granularity information.…”

Section: Hybrid Neural Networkmentioning

confidence: 99%

Implicit Discourse Relation Recognition via a BiLSTM-CNN Architecture With Dynamic Chunk-Based Max Pooling

Guo

Dang

2019

IEEE Access

View full text Add to dashboard Cite

Implicit discourse relation recognition is a serious challenge in discourse analysis, which aims to understand and annotate the latent relations between two discourse arguments, such as temporal and comparison. Most neural network-based models encode linguistic features (such as syntactic parsing and position information) as embedding vectors, which are prone to error propagation due to unsuitable pre-processing. Other methods apply different attention or memory mechanisms, mainly considering the key points in the discourse, yet ignore some valuable clues. In particular, those using convolution neural networks retain local contexts but lose word order information due to the standard pooling operation. The methods that use bidirectional long short-term memory network consider the word sequence and retain the global information, but cannot capture the context with different range sizes. In this paper, we propose a novel Dynamic Chunk-based Max Pooling BiLSTM-CNN framework (DC-BCNN) to address these issues. First, we exploit BiLSTMs to capture the semantic representations of discourse arguments. Second, we adopt the proposed convolutional layer to automatically extract the ''multi-granularity'' features (just like n-gram) by setting different convolution filter sizes. Then, we design a dynamic chunk-based max pooling strategy to obtain the important scaled features of different parts in one discourse argument. This strategy can dynamically divide each argument into several segments (called chunks) according to the argument length and the number of current pooling layer in the CNN and then select the maximum value of each chunk to indicate crucial information. We further utilize a fully connected layer with a softmax function to recognize discourse relations. The experimental results on two corpora (i.e., PDTB and HIT-CDTB) show that our proposed model is effective in implicit discourse relation recognition. INDEX TERMS Implicit discourse relation recognition, discourse argument representation, dynamic chunkbased max pooling, BiLSTM, CNN.

show abstract

Memory Augmented Attention Model for Chinese Implicit Discourse Relation Recognition

Cited by 5 publications

References 23 publications

A Survey of Implicit Discourse Relation Recognition

A Survey of Implicit Discourse Relation Recognition

Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

Implicit Discourse Relation Recognition via a BiLSTM-CNN Architecture With Dynamic Chunk-Based Max Pooling

Contact Info

Product

Resources

About