Two-Stage Attention-Based Model for Code Search with Textual and Structural Features

Xu, Lan; Yang, Huanhuan; Liu, Chao; Shuai, Jianhang; Yan, Meng; Lei, Yan; Zhou, Xu

doi:10.1109/saner50967.2021.00039

Cited by 36 publications

(28 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on DeepCS, Shuai et al [25] proposed a co-attentive representation learning model to capture the correlations between query and code. Xu et al [31] introduced a two-stage attention-based model to learn the semantic correlation effectively and efficiently.…”

Section: Code Searchmentioning

confidence: 99%

CSRS: Code Search with Relevance Matching and Semantic Matching

Cheng¹,

Kuang²

2022

Preprint

View full text Add to dashboard Cite

Developers often search and reuse existing code snippets in the process of software development. Code search aims to retrieve relevant code snippets from a codebase according to natural language queries entered by the developer. Up to now, researchers have already proposed information retrieval (IR) based methods and deep learning (DL) based methods. The IR-based methods focus on keyword matching, that is to rank codes by relevance between queries and code snippets, while DL-based methods focus on capturing the semantic correlations. However, the existing methods do not consider capturing two matching signals simultaneously. Therefore, in this paper, we propose CSRS, a code search model with relevance matching and semantic matching. CSRS comprises (1) an embedding module containing convolution kernels of different sizes which can extract n-gram embeddings of queries and codes, (2) a relevance matching module that measures lexical matching signals, and (3) a co-attention based semantic matching module to capture the semantic correlation. We train and evaluate CSRS on a dataset with 18.22M and 10k code snippets. The experimental results demonstrate that CSRS achieves an MRR of 0.614, which outperforms two state-of-the-art models DeepCS and CARLCS-CNN by 33.77% and 18.53% respectively. In addition, we also conducted several experiments to prove the effectiveness of each component of CSRS. CCS CONCEPTS• Software and its engineering → Reusability; • Information systems → Retrieval models and ranking.

show abstract

Section: Code Searchmentioning

confidence: 99%

CSRS: Code Search with Relevance Matching and Semantic Matching

Cheng¹,

Kuang²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, the method CARLCS-CNN [24] introduces a co-attentive mechanism on the base of DeepCS architecture; Yan. et al [29] designed a two-stage-attention-based model for their TAB-CS model. Such attention-based models have improved the primordial DeepCS to better capture the long-range relationship between tokens, thereby achieving a performance superior to DeepCS.…”

Section: Background and Related Work 21 Neural Code Searchmentioning

confidence: 99%

“…After deduplication, there are 48K records remaining. The experimental results in Section 5 demonstrate MM-SCS outperforms four state of-the-art models UNIF [7], DeepCS [10], CARLCS-CNN [24], and TAB-CS [29] by 34.2%, 59.3%, 36.8%, and 14.1% in terms of MRR, respectively. In addition, MM-SCS achieves 0.34s/query in the testing process, which is only second to UNIF.…”

Section: Introductionmentioning

confidence: 98%

“…MM-SCS adopts multi-head self-attention networks [25] to embed three textual modalities (code tokens, function name, and API sequence), and a modified graph attention network [20] to embed CEDG, respectively, introduced in Section 4.2. Compared with existing models with no attention mechanism [10] or with only single headed attention networks [7,24,29], multi-head attention mechanism can effectively learn different features from different heads and emphasize important features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Code Search for Smart Contracts

Shi¹,

Xiang²,

Yu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Semantic code search technology allows searching for existing code snippets through natural language, which can greatly improve programming efficiency. Smart contracts, programs that run on the blockchain, have a code reuse rate of more than 90%, which means developers have a great demand for semantic code search tools. However, the existing code search models still have a semantic gap between code and query, and perform poorly on specialized queries of smart contracts. In this paper, we propose a Multi-Modal Smart contract Code Search (MM-SCS) model. Specifically, we construct a Contract Elements Dependency Graph (CEDG) for MM-SCS as an additional modality to capture the data-flow and control-flow information of the code. To make the model more focused on the key contextual information, we use a multi-head attention network to generate embeddings for code features. In addition, we use a finetuned pretrained model to ensure the model's effectiveness when the training data is small. We compared MM-SCS with four stateof-the-art models on a dataset with 470K (code, docstring) pairs collected from Github and Etherscan. Experimental results show that MM-SCS achieves an MRR (Mean Reciprocal Rank) of 0.572, outperforming four state of-the-art models UNIF, DeepCS, CARLCS-CNN, and TAB-CS by 34.2%, 59.3%, 36.8%, and 14.1%, respectively. Additionally, the search speed of MM-SCS is second only to UNIF, reaching 0.34s/query. CCS CONCEPTS• Software and its engineering → Search-based software engineering.

show abstract

“…Aimed to tackle certain issues in code search, CQIL [60] models the semantic correlations between code and query with hybrid representations. Similarly, NJACS [61], CARLCS [62], TabCS [63] and SANCS [64] learn attention-based representations of code and query with the co-attention mechanism.…”

Section: Text-based Code Searchmentioning

confidence: 99%

Multimodal Representation for Neural Code Search

Gu,

Chen,

Monperrus

2021

Preprint

View full text Add to dashboard Cite

Semantic code search is about finding semantically relevant code snippets for a given natural language query. In the state-of-the-art approaches, the semantic similarity between code and query is quantified as the distance of their representation in the shared vector space. In this paper, to improve the vector space, we introduce tree-serialization methods on a simplified form of AST and build the multimodal representation for the code data. We conduct extensive experiments using a single corpus that is large-scale and multi-language: CodeSearchNet. Our results show that both our tree-serialized representations and multimodal learning model improve the performance of code search. Last, we define intuitive quantification metrics oriented to the completeness of semantic and syntactic information of the code data, to help understand the experimental findings.

show abstract

Two-Stage Attention-Based Model for Code Search with Textual and Structural Features

Cited by 36 publications

References 45 publications

CSRS: Code Search with Relevance Matching and Semantic Matching

CSRS: Code Search with Relevance Matching and Semantic Matching

Semantic Code Search for Smart Contracts

Multimodal Representation for Neural Code Search

Contact Info

Product

Resources

About