Neural Attribute Machines for Program Generation

Amodio, Matthew; Chaudhuri, Swarat; Reps, Thomas

doi:10.48550/arxiv.1705.09231

Cited by 7 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This simple process always produces correct code, even when the network does not learn to produce it. In contrast, Amodio et al [16] create a significantly more complex model that aims to learn to enforce deterministic constraints of the code generation, rather than enforcing them on the directly on the output. We further discuss the issue of embedding constraints and problem structure in models vs. learning the constraints in Section 6.…”

Section: Code-generating Probabilistic Models Of Source Codementioning

confidence: 99%

A Survey of Machine Learning for Big Code and Naturalness

et al. 2018

View full text Add to dashboard Cite

Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit code's abundance of patterns. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of each model and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss crosscutting and application-specific challenges and opportunities.1 It may be worth pointing out that deep learning and probabilistic modeling are not mutually exclusive. Indeed, many of the currently most effective methods for language modeling, for example, are based on deep learning. of probabilistic source code models (Section 5). Finally, we mention a few overlapping research areas (Section 7), and we discuss challenges and interesting future directions (Section 6).Related Reviews and other Resources. There have been short reviews summarizing the progress and the vision of the research area, from both software engineering [52] and programming languages perspectives [28,195]. However, none of these articles can be considered extensive literature reviews, which is the purpose of this work. Ernst [57] discusses promising areas of applying natural language processing to software development, including error messages, variable names, code comments, and user questions. Some resources, datasets and code can be found at http://learnbigcode.github.io/. An online version of the work reviewed here -which we will keep up-to-date by accepting external contributions -can be found at https://ml4code.github.io. THE NATURALNESS HYPOTHESISMany aspects of code, such as names, formatting, the lexical order of methods, etc. have no impact on program semantics. This is precisely why we abstract them in most program analyses. But then, why should statistical properties of code matter at all? To explain this, we recently suggested a hypothesis, called the naturalness hypothesis. The inspiration for the naturalness hypothesis can be traced back to the "literate programming" concept of D. Knuth, which draws from the insight that programming is a form of human communication: "Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do... " [105] The naturalness hypothesis, then, holds thatThe naturalness hypothesis. Software is a form of human communication; software corpora have similar statistical properties to natural language corpora; and these properties can be exploited to build better software engineering tools.The exploitation of the statistics of human communication is a mature and effective technology, with numerous applications ...

show abstract

Section: Code-generating Probabilistic Models Of Source Codementioning

confidence: 99%

A Survey of Machine Learning for Big Code and Naturalness

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Of course, there has been a lot of other research in compiler fuzzing. Several works [28], [29] apply machine learning to improve the fuzzing performance. CodeAlchemist [30] is based on the idea of semantic-aware assembly of JavaScript programs.…”

Section: Related Workmentioning

confidence: 99%

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Stepanov¹,

Akhin²,

Belyaev³

2020

Preprint

View full text Add to dashboard Cite

Kotlin is a relatively new programming language from JetBrains: its development started in 2010 with release 1.0 done in early 2016. The Kotlin compiler, while slowly and steadily becoming more and more mature, still crashes from time to time on the more tricky input programs, not least because of the complexity of its features and their interactions. This makes it a great target for fuzzing, even the basic forms of which can find a significant number of Kotlin compiler crashes.There is a problem with fuzzing, however, closely related to the cause of the crashes: generating a random, non-trivial and semantically valid Kotlin program is hard. In this paper, we talk about type-centric compiler fuzzing in the form of typecentric enumeration, an approach inspired by skeletal program enumeration [1] and based on a combination of generative and mutation-based fuzzing, which solves this problem by focusing on program types. After creating the skeleton program, we fill the typed holes with fragments of suitable type, created via generation and enhanced by semantic-aware mutation.We implemented this approach in our Kotlin compiler fuzzing framework called Backend Bug Finder (BBF) and did an extensive evaluation, not only testing the real-world feasibility of our approach, but also comparing it to other compiler fuzzing techniques. The results show our approach to be significantly better compared to other fuzzing approaches at generating semantically valid Kotlin programs, while creating more interesting crash-inducing inputs at the same time. We managed to find more than 50 previously unknown compiler crashes, of which 18 were considered important after their triage by the compiler team.

show abstract

“…In an AST, a program is parsed into a hierarchy of non-terminal and terminal (leaf) nodes based on the syntax of a programming language. To utilize AST for code representation, the simplest way would be to use depth-first search to convert an AST into a sequence [12,54,149]. Other studies proposed DL models (Recursive Neural Networks [267], Tree-LSTM [263] or CNN [185]) to work directly on the hierarchical structure of a parse tree.…”

Section: Deep Encoder Modelsmentioning

confidence: 99%

“…Bash shell Lin et al 2017 [154] CSS and HTML Beltramelli 2017 [21] Seq2seq with grammar constraint Simple C code Amodio et al 2017 [12] Seq2AST Domain-specific language Dong and Lapata 2016 [61] Card-game code Reinforcement learning SQL queries Zhong et al 2017 [286] language to use input-output examples to create a complete program. Differentiable approaches often perform worse than the search-based methods for low-level programming languages (e.g., Assembly) [72].…”

Section: Seq2seqmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

Le,

Chen,

Babar

2020

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well.CCS Concepts: • General and reference → Surveys and overviews; • Computing methodologies → Neural networks; Natural language processing; • Software and its engineering → Software notations and tools; Source code generation;

show abstract

Neural Attribute Machines for Program Generation

Cited by 7 publications

References 19 publications

A Survey of Machine Learning for Big Code and Naturalness

A Survey of Machine Learning for Big Code and Naturalness

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

Contact Info

Product

Resources

About