Self-Attention Networks for Code Search

Fang, Sen; Tan, Youshuai; Zhang, Tao; Liu, Yepang

doi:10.1016/j.infsof.2021.106542

Cited by 39 publications

(37 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Related Workmentioning

confidence: 92%

“…Even though the latest DEEPCS's variants, such as CARLCS-CNN [39] and SAN-CS [6] claimed better performance, they bring additional restrictions to the compatibility with our experiments. Both CARLCS-CNN [39] and SAN-CS [6] introduce the co-attention mechanism to refine the code and query representations. Other than generating the independeny vector representations for the code snippet and query, they compute a joint attention representation, aiming at not only catching the semantic information, but also the semantic relation between the two parts.…”

Section: Related Workmentioning

confidence: 92%

“…It uses the co-attention method to build the semantic relationship between code snippets and related queries. Similarly, Fang et al [39] proposed SAN-CS, which is solely based on the self-attention method to achieve the same purpose.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

On the Effectiveness of Transfer Learning for Code Search

Salza,

Schwizer,

et al. 2021

Preprint

View full text Add to dashboard Cite

The Transformer architecture and transfer learning have marked a quantum leap in natural language processing, improving the state of the art across a range of text-based tasks. This paper examines how these advancements can be applied to and improve code search. To this end, we pre-train a BERT-based model on combinations of natural language and source code data and evaluate it on pairs of StackOverflow question titles and code answers. Our results show that the pre-trained models consistently outperform the models that were not pre-trained. In cases where the model was pre-trained on natural language "and" source code data, it also outperforms an information retrieval baseline based on Lucene. Also, we demonstrated that combined use of an information retrieval-based approach followed by a Transformer, leads to the best results overall, especially when searching into a large search pool. Furthermore, transfer learning is particularly effective when much pre-training data is available and fine-tuning data is limited. We demonstrate that natural language processing models based on the Transformer architecture can be directly applied to source code analysis tasks, such as code search. With the development of Transformer models designed more specifically for dealing with source code data, we believe the results on source code analysis tasks can be further improved.

show abstract

Section: Related Workmentioning

confidence: 92%

Section: Related Workmentioning

confidence: 92%

See 1 more Smart Citation

On the Effectiveness of Transfer Learning for Code Search

Salza,

Schwizer,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…SuccessRate@k is widely used by many previous studies (Haldar et al, 2020;Shuai et al, 2020;Fang et al, 2021;Heyman and Cutsem, 2020). The metric is calculated as follows:…”

Section: Evaluation Metricmentioning

confidence: 99%

Accelerating Code Search with Deep Hashing and Code Classification

Gu¹,

Wang²,

Du³

et al. 2022

Preprint

View full text Add to dashboard Cite

Code search is to search reusable code snippets from source code corpus based on natural languages queries. Deep learning-based methods on code search have shown promising results. However, previous methods focus on retrieval accuracy, but lacked attention to the efficiency of the retrieval process. We propose a novel method CoSHC to accelerate code search with deep hashing and code classification, aiming to perform efficient code search without sacrificing too much accuracy. To evaluate the effectiveness of CoSHC, we apply our method on five code search models. Extensive experimental results indicate that compared with previous code search baselines, CoSHC can save more than 90% of retrieval time meanwhile preserving at least 99% of retrieval accuracy.

show abstract

“…Aimed to tackle certain issues in code search, CQIL [60] models the semantic correlations between code and query with hybrid representations. Similarly, NJACS [61], CARLCS [62], TabCS [63] and SANCS [64] learn attention-based representations of code and query with the co-attention mechanism.…”

Section: Text-based Code Searchmentioning

confidence: 99%

Multimodal Representation for Neural Code Search

Gu,

Chen,

Monperrus

2021

Preprint

View full text Add to dashboard Cite

Semantic code search is about finding semantically relevant code snippets for a given natural language query. In the state-of-the-art approaches, the semantic similarity between code and query is quantified as the distance of their representation in the shared vector space. In this paper, to improve the vector space, we introduce tree-serialization methods on a simplified form of AST and build the multimodal representation for the code data. We conduct extensive experiments using a single corpus that is large-scale and multi-language: CodeSearchNet. Our results show that both our tree-serialized representations and multimodal learning model improve the performance of code search. Last, we define intuitive quantification metrics oriented to the completeness of semantic and syntactic information of the code data, to help understand the experimental findings.

show abstract

Self-Attention Networks for Code Search

Cited by 39 publications

References 32 publications

On the Effectiveness of Transfer Learning for Code Search

On the Effectiveness of Transfer Learning for Code Search

Accelerating Code Search with Deep Hashing and Code Classification

Multimodal Representation for Neural Code Search

Contact Info

Product

Resources

About