MaterialGAN

We propose a method to accelerate the joint process of physically acquiring and learning neural Bi‐directional Reflectance Distribution Function (BRDF) models. While BRDF learning alone can be accelerated by meta‐learning, acquisition remains slow as it relies on a mechanical process. We show that meta‐learning can be extended to optimize the physical sampling pattern, too. After our method has been meta‐trained for a set of fully‐sampled BRDFs, it is able to quickly train on new BRDFs with up to five orders of magnitude fewer physical acquisition samples at similar quality. Our approach also extends to other linear and non‐linear BRDF models, which we show in an extensive evaluation.

Section: Previous Workmentioning

confidence: 99%

Learning to Learn and Sample BRDFs

Liu

Fischer

Ritschel

2023

“…Such latent‐space optimization is not possible with methods based on the UNet convolutional neural network [TWPH20], as the skip connections transmit shape information while bypassing the encoder‐decoder bottleneck. Latent‐space optimization has proven to be a very effective regularization strategy for other ill‐posed inverse problems, such as material recovery from few photographs [GLD*19, GSH*20] and shape completion from partial point clouds [PFS*19].…”

Section: Related Workmentioning

confidence: 99%

Interactive design of 2D car profiles with aerodynamic feedback

Rosset

Cordonnier

Duvigneau

et al. 2023

The design of car shapes requires a delicate balance between aesthetic and performance. While fluid simulation provides the means to evaluate the aerodynamic performance of a given shape, its computational cost hinders its usage during the early explorative phases of design, when aesthetic is decided upon. We present an interactive system to assist designers in creating aerodynamic car profiles. Our system relies on a neural surrogate model to predict fluid flow around car shapes, providing fluid visualization and shape optimization feedback to designers as soon as they sketch a car profile. Compared to prior work that focused on time‐averaged fluid flows, we describe how to train our model on instantaneous, synchronized observations extracted from multiple pre‐computed simulations, such that we can visualize and optimize for dynamic flow features, such as vortices. Furthermore, we architectured our model to support gradient‐based shape optimization within a learned latent space of car profiles. In addition to regularizing the optimization process, this latent space and an associated encoder‐decoder allows us to input and output car profiles in a bitmap form, without any explicit parameterization of the car boundary. Finally, we designed our model to support pointwise queries of fluid properties around car shapes, allowing us to adapt computational cost to application needs. As an illustration, we only query our model along streamlines for flow visualization, we query it in the vicinity of the car for drag optimization, and we query it behind the car for vortex attenuation.

“…There are a multitude of deep generative modeling paradigms [BTLLW22, CHIS22], all with different pros and cons: generative adversarial networks (GANs), variational autoencoders (VAEs), normalizing flows, auto‐regressive models, and diffusion models. These deep generative models have been applied across many visual domains such as natural images [KLA21], materials [GSH*20], sketches [VPB*22], scenes [WSCR18], voxels [WZX*16], meshes [NGEB20], implicit shapes [CZ19], and character motion [LZCvdP20]. As in the reconstruction context, many of these approaches have been designed to produce visual outputs directly, without any intermediate symbolic representation, and thus are outside the scope of what our survey covers.…”

Section: Background and Scopementioning

confidence: 99%

Neurosymbolic Models for Computer Graphics

Ritchie

Guerrero

Jones

et al. 2023

Procedural models (i.e. symbolic programs that output visual data) are a historically‐popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high‐quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI‐based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state‐of‐the‐art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.