DRr-Net: Dynamic Re-Read Network for Sentence Semantic Matching

Zhang, Kun; Lv, Guangyi; Wang, Linyuan; Wu, Le; Chen, Enhong; Wu, Fangzhao; Xie, Xing

doi:10.1609/aaai.v33i01.33017442

Cited by 33 publications

(18 citation statements)

References 37 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[23]. At the same time, the traditional capsule network is improved, words that have nothing to do with text semantics are regarded as noise capsules, and smaller weights are assigned to reduce the impact on subsequent tasks [24]. The text semantic matching method first uses the pretrained GloVe model to map the two texts into a 300dimensional word vector matrix [25].…”

Section: Online Trading Platform Text Semanticmentioning

confidence: 99%

Semantic Matching Efficiency of Supply and Demand Text on Cross‐Border E‐Commerce Online Technology Trading Platforms

Chen

2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the innovation of global trade business models, more and more foreign trade companies are transforming and developing in the direction of cross-border e-commerce. However, due to the limitation of platform language processing and analysis technology, foreign trade companies encounter many bottlenecks in the process of transformation and upgrading. From the perspective of the semantic matching efficiency of e-commerce platforms, this paper improves the logical and technical problems of cross-border e-commerce in the operation process and uses semantic matching efficiency as the research object to conduct experiments on the QQP dataset. We propose a graph network text semantic analysis model TextSGN based on semantic dependency analysis for the problem that the existing text semantic matching method does not consider the semantic dependency information between words in the text and requires a large amount of training data. The model first analyzes the semantic dependence of the text and performs word embedding and one-hot encoding on the nodes (single words) and edges (dependencies) in the semantic dependence graph. On this basis, in order to quickly mine semantic dependencies, an SGN network block is proposed. The network block defines the way of information transmission from the structural level to update the nodes and edges in the graph, thereby quickly mining semantics dependent information allows the network to converge faster, train classification models on multiple public datasets, and perform classification tests. The experimental results show that the accuracy rate of TextSGN model in short text classification reaches 95.2%, which is 3.6% higher than the suboptimal classification method; the accuracy rate is 86.16%, the F 1 value is 88.77%, and the result is better than other methods.

show abstract

Section: Online Trading Platform Text Semanticmentioning

confidence: 99%

Semantic Matching Efficiency of Supply and Demand Text on Cross‐Border E‐Commerce Online Technology Trading Platforms

Chen

2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…For these methods, missing semantics is still a difficult issue. To address this problem, researchers discovered that the granularity of text is also crucial for capturing semantic information [4][5][6][7][8], especially for Chinese SSM. In particular, the MGF model [7] integrates character/word granularity and achieves remarkable results.…”

Section: Related Workmentioning

confidence: 99%

“…With the development of deep learning, such as the attention mechanism [1,2] and Siamese networks [3], it is possible to capture deep semantic information of sentences. In English SSM tasks, [4,5] lead the way in using multi-granularity to extract fine-grained information. Although the character-granularity is beneficial to enrich English text representation, one single English character does not express meaning.…”

Section: Introductionmentioning

confidence: 99%

Multi-Granularity and Internal-External Correlation Residual Model for Chinese Sentence Semantic Matching

Zhang

Chen

2021

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Sentence semantic matching (SSM) is central to many natural language processing tasks. This is especially the case for Chinese sentence semantic matching due to the complexity of the semantics, missing semantics and semantic confusion are more likely to occur. Existing methods have used enhanced text representations and multiple matching strategies to address these problems but there is still great potential to capture deep semantic information for Chinese text. This paper proposes a Multi-Granularity and Internal-External correlation Residual model (MGIER) to better capture the deep semantic information and to alleviate the missing semantics effectively. First, the MGIER model utilizes character/word granularity to capture fine-grained information. Then, soft alignment attention is employed to enhance the correlation between characters/words in a sentence, called internal correlation, and the correlation between sentences, called external correlation. In particular, this method uses residual connections to preserve more semantic information from the bottom embedding layer to the top prediction layer. Experimental results show that the proposed method achieves state-of-the-art performance for Chinese SSM, and, compared with pre-trained models, the method also achieves better performance with fewer parameters.

show abstract

“…In our preliminary work [26], we leverage DRr unit to select one important word at each reading step. However, one word can express tremendously different semantic meanings due to different local structures.…”

Section: Dynamic Sequential Attention (Dsa)mentioning

confidence: 99%

“…In order to solve the challenge of how to employ attention mechanism in a dynamic way for enhancing its ability of selecting important parts, in our preliminary work [26], we proposed the Dyanmic Re-read (DRr) attention, a novel architecture that selects one important word at each reading step and reads the important words repeatedly for precise sentence semantic representation. Based on this novel attention, we developed a Dynamic Re-read Network (DRr-Net) for sentence semantic matching, in which Global Sentence Encoding is used to model sentences comprehensively and DRr unit is used to capture the important parts precisely for better sentence semantic representation and matching.…”

Section: Introductionmentioning

confidence: 99%

LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching

Zhang,

Lv,

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, which is widely used in various natural language tasks, such as Natural Language Inference (NLI), Paraphrase Identification (PI), and so on. Much recent progress has been made in this area, especially attention-based methods and pre-trained language model based methods. However, most of these methods focus on all the important parts in sentences in a static way and only emphasize how important the words are to the query, inhibiting the ability of attention mechanism. In order to overcome this problem and boost the performance of attention mechanism, we propose a novel dynamic re-read attention, which can pay close attention to one small region of sentences at each step and re-read the important parts for better sentence representations. Based on this attention variation, we develop a novel Dynamic Re-read Network (DRr-Net) for sentence semantic matching. Moreover, selecting one small region in dynamic re-read attention seems insufficient for sentence semantics, and employing pre-trained language models as input encoders will introduce incomplete and fragile representation problems. To this end, we extend DRr-Net to Locally-Aware Dynamic Re-read Attention Net (LadRa-Net), in which local structure of sentences is employed to alleviate the shortcoming of Byte-Pair Encoding (BPE) in pre-trained language models and boost the performance of dynamic reread attention. Extensive experiments on two popular sentence semantic matching tasks demonstrate that DRr-Net can significantly improve the performance of sentence semantic matching. Meanwhile, LadRa-Net is able to achieve better performance by considering the local structures of sentences. In addition, it is exceedingly interesting that some discoveries in our experiments are consistent with some findings of psychological research.

show abstract

DRr-Net: Dynamic Re-Read Network for Sentence Semantic Matching

Cited by 33 publications

References 37 publications

Semantic Matching Efficiency of Supply and Demand Text on Cross‐Border E‐Commerce Online Technology Trading Platforms

Semantic Matching Efficiency of Supply and Demand Text on Cross‐Border E‐Commerce Online Technology Trading Platforms

Multi-Granularity and Internal-External Correlation Residual Model for Chinese Sentence Semantic Matching

LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching

Contact Info

Product

Resources

About