Simple and Effective Text Matching with Richer Alignment Features

Yang, Runqi; Zhang, Jianhai; Gao, Xing; Ji, Feng; Chen, Haiqing

doi:10.48550/arxiv.1908.00300

Cited by 18 publications

(33 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Where O l represents the output of the l th layer H l , and I l represents the input of the layer H l , and [; ] denotes the concatenation operation. Unlike the previous layers, for the Matching Composition Layer's input, we refer to the idea of Yang et al [26], adopt another version of the residual connection. As shown in Figure 3, between the Matching Composition Layer and the Fusion Layer, the Attention Layer's output is not connected.…”

Section: Shortcut Connectionsmentioning

confidence: 99%

“…In the Fusion Layer, the contextual representation and the Inter-Attention representation of word are integrated to fuse aligned features. We refer to the fusion method in the paper [26] and use the following three ways to fuse the features:…”

Section: F Fusion Layermentioning

confidence: 99%

“…Where − denotes calculate the difference between vectors, • represents element-wise multiplication, and [; ] refers to the concatenation operation. The subtraction operator highlights the difference between two vectors, while the multiplication highlights the similarity between the vectors [26]. F 1, F 2, F 3 are three single-layer feed-forward networks with independent parameters.…”

Section: F Fusion Layermentioning

confidence: 99%

“…Then we use a method similar to the paper [26], [30] to concatenate the original vector, the difference between the two vectors, and the result of element-wise multiplication, and input it into a multi-layer feed-forward network L to get the feature vector:…”

Section: H Prediction Layermentioning

confidence: 99%

See 3 more Smart Citations

Attention-based model for predicting question relatedness on Stack Overflow

Pei

Qin

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

Stack Overflow is one of the most popular Programming Community-based Question Answering (PCQA) websites that has attracted more and more users in recent years. When users raise or inquire questions in Stack Overflow, providing related questions can help them solve problems. Although there are many approaches based on deep learning that can automatically predict the relatedness between questions, those approaches are limited since interaction information between two questions may be lost. In this paper, we adopt the deep learning technique, propose an Attention-based Sentence pair Interaction Model (ASIM) to predict the relatedness between questions on Stack Overflow automatically. We adopt the attention mechanism to capture the semantic interaction information between the questions. Besides, we have pre-trained and released word embeddings specific to the software engineering domain for this task, which may also help other related tasks. The experiment results demonstrate that ASIM has made significant improvement over the baseline approaches in Precision, Recall, and Micro-F1 evaluation metrics, achieving state-of-the-art performance in this task. Our model also performs well in the duplicate question detection task of AskUbuntu, which is a similar but different task, proving its generalization and robustness.

show abstract

Section: Shortcut Connectionsmentioning

confidence: 99%

Section: F Fusion Layermentioning

confidence: 99%

Section: F Fusion Layermentioning

confidence: 99%

Section: H Prediction Layermentioning

confidence: 99%

See 2 more Smart Citations

Attention-based model for predicting question relatedness on Stack Overflow

Pei

Qin

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

show abstract

“…Pang et al [31] evaluated the question similarity from hierarchical levels. Yang et al [49] built RE2 model with stacked alignment layers to keep the model fast while still yielding strong performance. Furthermore, many works [1,13,46] considered the use of different kinds of complementary information, such as question category, Wikipedia concepts and corresponding answers, for the question retrieval task.…”

Section: Question Retrievalmentioning

confidence: 99%

A Discriminative Semantic Ranker for Question Retrieval

Cai

Fan

Guo

et al. 2021

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval

View full text Add to dashboard Cite

Similar question retrieval is a core task in community-based question answering (CQA) services. To balance the effectiveness and efficiency, the question retrieval system is typically implemented as multi-stage rankers: The first-stage ranker aims to recall potentially relevant questions from a large repository, and the latter stages attempt to re-rank the retrieved results. Most existing works on question retrieval mainly focused on the re-ranking stages, leaving the first-stage ranker to some traditional term-based methods. However, term-based methods often suffer from the vocabulary mismatch problem, especially on short texts, which may block the re-rankers from relevant questions at the very beginning. An alternative is to employ embedding-based methods for the first-stage ranker, which compress texts into dense vectors to enhance the semantic matching. However, these methods often lose the discriminative power as term-based methods, thus introduce noise during retrieval and hurt the recall performance. In this work, we aim to tackle the dilemma of the first-stage ranker, and propose a discriminative semantic ranker, namely DenseTrans, for highrecall retrieval. Specifically, DenseTrans is a densely connected Transformer, which learns semantic embeddings for texts based on Transformer layers. Meanwhile, DenseTrans promotes low-level features through dense connections to keep the discriminative power of the learned representations. DenseTrans is inspired by DenseNet in computer vision (CV), but poses a new way to use the dense connectivity which is totally different from its original design purpose. Experimental results over two question retrieval benchmark datasets show that our model can obtain significant gain on recall against strong term-based methods as well as stateof-the-art embedding-based methods.

show abstract

Semantic Sentence Matching Based on Multiple Parallelly Organized Interaction Layers at Various Granularity Combinations With Two-Stage Aggregation Strategy

Wang,

2023

IEEE Access

View full text Add to dashboard Cite

Semantic sentence matching plays an essential role in resolving many problems in natural language processing (NLP) field, it has gained increasing research focus and shown great improvements in recent years. However, most currently existing researches are for English sentence matching, research on Chinese semantic matching are relatively less. Moreover, due to the rather complicated contextual expressions and grammatical structure of Chinese language, many existing models are still unable to quite effectively capture interaction information between sentences. Thus, in this work, we attempt to propose a novel deep model to better address Chinese semantic sentence matching. Specifically, the convolutional neural networks with various kernel sizes are first employed for the multi-granular contextual encoding of sentences, combined with multiple different cross-sentence alignment mechanisms, the semantic interactions can be more clearly and profoundly performed at various granularity combinations between sentences. Additionally, rather than serially stacking multiple interaction layers, we organize multiple interaction layers in a parallel manner, and by further introduction of attention pooling, the semantically aligned sentence attentive vectors would be adaptively aggregated from both perspectives of alignment mechanisms and granularity combinations, thus more stable and effective sentence interactive features can be extracted while attempting to alleviate potential sentence alignment error propagation issue existed in hierarchically stacked interaction structure. Finally, extensive experiments are conducted to evaluate the performance of our model, the experimental results demonstrate that our proposed approach outperforms many state-of-the-art models on sentence matching and is capable of gaining a more accurate understanding of semantic relationships between Chinese sentences.

show abstract

Simple and Effective Text Matching with Richer Alignment Features

Cited by 18 publications

References 24 publications

Attention-based model for predicting question relatedness on Stack Overflow

Attention-based model for predicting question relatedness on Stack Overflow

A Discriminative Semantic Ranker for Question Retrieval

Semantic Sentence Matching Based on Multiple Parallelly Organized Interaction Layers at Various Granularity Combinations With Two-Stage Aggregation Strategy

Contact Info

Product

Resources

About