Generative models are now used to create a variety of high-quality digital artifacts. Yet their use in designing physical objects has received far less attention. In this paper, we advocate for the construction toy, LEGO, as a platform for developing generative models of sequential assembly. We develop a generative model based on graph-structured neural networks that can learn from human-built structures and produce visually compelling designs. Our code is released at: https://github. com/uoguelph-mlrg/GenerativeLEGO.Generative models can now create high-quality and diverse digital artifacts, particularly in the domain of images [2] and text [3]. Generative models are also being applied to physical design, such as computational chemistry and materials but they are less prevalent in architecture, landscape design, or manufacturing. Motivated by the opportunity of generative design of our physical environment, in this paper we explore the use of generative models to modular physical design.Advances in generative models in areas such as images and text is in part due to the availability of data and ease of experimentation. Other areas of machine learning, such as reinforcement learning, have benefited from the availability of simulators. Simulators are conducive to experimentation supporting algorithm development and, at the same time, entertaining and familiar (c.f. the Arcade Learning Environment [4]). In the context of physical design, specifically the sequential assembly of modular structures, we believe that there is an analogue: the plastic construction toy, LEGO. Structures built from LEGO have a number of interesting properties: (i) they are complex enough to approach real-world design; ii) they are familiar and fun enough to attract interest from the generative models community; and iii) there is a huge amount of official and user-created structures available as a source of training data.Learning to sequentially assemble LEGO structures based on human-generated examples is a sequential decision making problem. It can be approached through a number of formalisms familiar to the ML community, including reinforcement learning [5], imitation learning [6], evolutionary design [7,8], and Bayesian optimization [9]. We are unaware of the use of deep generative models for LEGO. However, generative graph models (GGMs), an emerging sub-field of graph representation learning [10], are particularly well suited to this domain.Workshop on machine learning for engineering modeling, simulation and design @ NeurIPS 2020

No abstract

In image generation, generative models can be evaluated naturally by visually inspecting model outputs. However, this is not always the case for graph generative models (GGMs), making their evaluation challenging. Currently, the standard process for evaluating GGMs suffers from three critical limitations: i) it does not produce a single score which makes model selection challenging, ii) in many cases it fails to consider underlying edge and node features, and iii) it is prohibitively slow to perform. In this work, we mitigate these issues by searching for scalar, domain-agnostic, and scalable metrics for evaluating and ranking GGMs.To this end, we study existing GGM metrics and neural-network-based metrics emerging from generative models of images that use embeddings extracted from a task-specific network. Motivated by the power of certain Graph Neural Networks (GNNs) to extract meaningful graph representations without any training, we introduce several metrics based on the features extracted by an untrained random GNN. We design experiments to thoroughly test metrics on their ability to measure the diversity and fidelity of generated graphs, as well as their sample and computational efficiency. Depending on the quantity of samples, we recommend one of two random-GNN-based metrics that we show to be more expressive than pre-existing metrics. While we focus on applying these metrics to GGM evaluation, in practice this enables the ability to easily compute the dissimilarity between any two sets of graphs regardless of domain. Our code is released at: https://github.com/uoguelph-mlrg/GGM-metrics.

No abstract