Programs evolved by genetic programming unfortunately oen do not generalize to unseen data. Reliable synthesis of programs that generalize to unseen data is therefore an important open problem. We present evidence that smaller programs evolved using the PushGP system tend to generalize beer over a range of program synthesis problems. Like in many genetic programming systems, programs evolved by PushGP usually have pieces that can be removed without changing the behavior of the program. We describe methods for automatically simplifying evolved programs to make them smaller and potentially improve their generalization. We present ve simplication methods and analyze their strengths and weaknesses on a suite of general program synthesis benchmark problems. All of our methods use a straightforward hill-climbing procedure to remove pieces of a program while ensuring that the resulting program gives the same errors on the training data as did the original program. We show that automatic simplication, previously used both for post-run analysis and as a genetic operator, can signicantly improve the generalization rates of evolved programs.
A variety of inductive program synthesis (IPS) techniques have recently been developed, emerging from di erent areas of computer science. However, these techniques have not been adequately compared on general program synthesis problems. In this paper we compare several methods on problems requiring solution programs to handle various data types, control structures, and numbers of outputs. e problem set also spans levels of abstraction; some would ordinarily be approached using machine code or assembly language, while others would ordinarily be approached using highlevel languages. e presented comparisons are focused on the possibility of success; that is, on whether the system can produce a program that passes all tests, for all training and unseen testing inputs. e compared systems are Flash Fill, MagicHaskeller, TerpreT, and two forms of genetic programming. e two genetic programming methods chosen were PushGP and Grammar Guided Genetic Programming. e results suggest that PushGP and, to an extent, TerpreT and Grammar Guided Genetic Programming are more capable of nding solutions than the others, albeit at a higher computational cost. A more salient observation is the di culty of comparing these methods due to drastically di erent intended applications, despite the common goal of program synthesis. CCS CONCEPTS •So ware and its engineering → Genetic programming; • eory of computation → Evolutionary algorithms;
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.