DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning

Ellis, Kevin; Wong, Catherine C. L.; Nye, Maxwell; Sable-Meyer, Mathias; Cary, Luc; Lucas, Morales,; Hewitt, Luke; Solar-Lezama, Armando; Tenenbaum, Joshua B.

doi:10.48550/arxiv.2006.08381

Cited by 35 publications

(64 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Relying on the built-in method for program search, it allows us to efficiently search promising type-safe differentiable program candidates. Compared to other program synthesis languages [48][49][50][51] , HOUDINI rules out the error-prone functions that undermine the software safety and presents itself as an ideal candidate for our task (See Supplementary Tab. 7 for further comparison).…”

Section: Type-safe Program Synthesismentioning

confidence: 99%

See 1 more Smart Citation

Automated causal inference in application to randomized controlled clinical trials

Wu¹,

Horeweg²,

Bruyn³

et al. 2022

Preprint

View full text Add to dashboard Cite

Randomized controlled trials (RCTs) are considered as the gold standard for testing causal hypotheses in the clinical domain. However, the investigation of prognostic variables of patient outcome in a hypothesized cause-effect route is not feasible using standard statistical methods. Here, we propose a new automated causal inference method (AutoCI) built upon the invariant causal prediction (ICP) framework for the causal re-interpretation of clinical trial data. Compared to existing methods, we show that the proposed AutoCI allows to efficiently determine the causal variables with a clear differentiation on two real-world RCTs of endometrial cancer patients with mature outcome and extensive clinicopathological and molecular data. This is achieved via suppressing the causal probability of non-causal variables by a wide margin. In ablation studies, we further demonstrate that the assignment of causal probabilities by AutoCI remain consistent in the presence of confounders. In conclusion, these results confirm the robustness and feasibility of AutoCI for future applications in real-world clinical analysis.

show abstract

Section: Type-safe Program Synthesismentioning

confidence: 99%

“…Task Typed Functional Code availability NTPT 48 Misc. NS-CL 49 VQA Prob-NMN 50 VQA DreamCoder 51 Misc. HOUDINI 42 Misc.…”

Section: Data Availabilitymentioning

confidence: 99%

Automated causal inference in application to randomized controlled clinical trials

Wu¹,

Horeweg²,

Bruyn³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Multiple works train a neural network and then analyze it to get information, such as inferring missing objects via back-propagation descent [2], inspecting its gradients to guide a program synthesis approach [51], learning a blue-print with a GNN [12], or leveraging overparametrized-but-regularized auto-encoders [52]. Others, such as DreamCoder [16], take the explicit approach with a neural guided synthesis over the space of formulas. Unlike these works, our method can readily be applied both to symbolic conservation laws and to neural network conservation losses applied to raw videos.…”

Section: Related Workmentioning

confidence: 99%

“…It is worth noting that other datasets from Greydanus et al [22] (such as a planetary system) could not be included because the DSL required too much depth to reach the conserved energy quantity. A better search, such as using evolution, or a better DSL (such as those derived from many scientific formulas in DreamCoder [16]) could remedy this. Finally, note that an approach that searched over the same DSL encoding the loss as a generic f (x 0 , x t ) loss, not as a conservation |g(x 0 ) − g(x t )| would require more than twice the depth, thus not being able to cover the evaluated datasets.…”

Section: Experimental Details For Scientific Datamentioning

confidence: 99%

Noether Networks: Meta-Learning Useful Conserved Quantities

Alet¹,

Doblar²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether's theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.

show abstract

“…This has led to a number of interesting applications, and deep-learning models are now successfully and routinely applied in tools that assist developers in writing and understanding programs and code. For instance, neural language models can synthesize (Gulwani et al, 2017;Ellis et al, 2020), complete (Chen et al, 2021), and summarize programs (Elnaggar et al, 2021), whether they are written in mainstream languages like Python and Java, or in domain-specific languages like SQL and regex.…”

Section: Introductionmentioning

confidence: 99%

Towards Neural Functional Program Evaluation

Scholak¹,

Pilault²,

Velez-Ginorio³

2021

Preprint

View full text Add to dashboard Cite

This paper explores the capabilities of current transformer-based language models for program evaluation of simple functional programming languages. We introduce a new program generation mechanism that allows control over syntactic sugar for semantically equivalent programs. T5 experiments reveal that neural functional program evaluation performs surprisingly well, achieving high 90% exact program match scores for most in-distribution and out-of-distribution tests. Using pretrained T5 weights has significant advantages over random initialization. We present and evaluate on three datasets to study generalization abilities that are specific to functional programs based on: type, function composition, and reduction steps. Code and data are publicly available at https://github.com/ElementAI/neural-interpreters.

show abstract

DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning

Cited by 35 publications

References 17 publications

Automated causal inference in application to randomized controlled clinical trials

Automated causal inference in application to randomized controlled clinical trials

Noether Networks: Meta-Learning Useful Conserved Quantities

Towards Neural Functional Program Evaluation

Contact Info

Product

Resources

About