Disentangled Code Representation Learning for Multiple Programming Languages

Abstract

Developing effective distributed representations of source code is fundamental yet challenging for many software engineering tasks such as code clone detection, code search, code translation and transformation. However, current code embedding approaches that represent the semantic and syntax of code in a mixed way are less interpretable and the resulting embedding can not be easily generalized across programming languages. In this paper, we propose a disentangled code representation learning approach to separate the semantic from the syntax of source code under a multi-programming-language setting, obtaining better interpretability and generalizability. Specially, we design three losses dedicated to the characteristics of source code to enforce the disentanglement effectively. We conduct comprehensive experiments on a real-world dataset composed of programming exercises implemented by multiple solutions that are semantically identical but grammatically distinguished. The experimental results validate the superiority of our proposed disentangled code representation, compared to several baselines, across three types of downstream tasks, i.e., code clone detection, code translation, and code-to-code search.

On the Effectiveness of Transfer Learning for Code Search

“…Instead, DYPRO [75] and LIGER [76] learn program representations through dynamic executions, from the mixture of symbolic and concrete execution traces. Zhang et al [77] addressed the problem of code representation using a multi-language setting to create an embedding that separates the semantic from the context of the source code. FLOW2VEC [78] is an embedding approach that preserves interprocedural program dependence by approximating the high-order proximity.…”

Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach

“…For this example, incorporating the caller/callee information may help boost the effectiveness. Beyond that, some more advanced static-analysis-based code embedding approaches (e.g., using data-flows and control-flows) [55], [65] would be beneficial to capture the code change semantics and thus deserve exploration in future studies.…”

Multilayer self‐attention residual network for code search

“…Wan et al 45 discussed the effectiveness of CodeBERT and GraphCodeBERT for preprocessing in code characterization, respectively. Zhang et al 46 separated the semantics from the syntax of the source code semantics from the syntax of the source code that obtains better interpretability and generality. Sui et al 47 precisely preserved the program dependencies (also known as value streams) between programs embedded in the control flow and alias-aware data flow of programs into a low-dimensional vector space by approximating higher-order proximity, and proposed the Flow2Vec model.…”