Neural Cellular Automata Manifold

Ruiz, Alejandro Hernandez; Vilalta, Armand; Moreno-Noguer, Francesc

doi:10.1109/cvpr46437.2021.00989

Cited by 5 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such an improved embedding can be extended multiple entities. Another approach would be to adopt the autoencoder from Ruiz et al (2021) to encode entities' structural information.…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…Instead of pre-defined, the update rules are neural network outputs from a Markov Chain. As Ruiz et al (2021), this enables the model to generate multiple structures, contrary to the typical NCA, which is only specific to one target. One limitation is that there is only one type of building block.…”

Section: Neural Cellular Automatamentioning

confidence: 99%

“…Compared to previous literature, our work has higher dimensionality (3D instead of 2D) than Mordvintsev et al (2020) and Ruiz et al (2021), more construction unit types than Zhang et al (2021), and we capture all surrounding cells with a 3D convolution instead of only the four immediate neighbors as in Horibe et al (2021).…”

Section: Neural Cellular Automatamentioning

confidence: 99%

“…The sophisticated nature of morphogenesis makes NCAs a promising modeling approach. Works on simulating morphogenesis with NCAs have primarily focused on easilyreplicable artefacts, such as pixelated emojis (Mordvintsev et al, 2020) and 2D images (Ruiz et al, 2021). Recent progress has also demonstrated its capability in generating high-fidelity 3D shapes (Zhang et al, 2021) and regenerating damaged morphology (Horibe et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Shyam¹,

Grbic²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems.

show abstract

“…Such an improved embedding can be extended multiple entities. Another approach would be to adopt the autoencoder from Ruiz et al (2021) to encode entities' structural information.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Section: Neural Cellular Automatamentioning

confidence: 99%

Section: Neural Cellular Automatamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Shyam¹,

Grbic²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Inspired by the image inpainting techniques, Hernandez et al . [RGM19] reformulate motion prediction as an inpainting task to complete the masked joints in spatiotemporal volumes. It is well‐known that recurrent neural networks (RNNs) have inherent advantages in modeling sequential data.…”

Section: Related Workmentioning

confidence: 99%

Diverse Dance Synthesis via Keyframes with Transformer Controllers

Pan

Wang

Bai

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

Existing keyframe-based motion synthesis mainly focuses on the generation of cyclic actions or short-term motion, such as walking, running, and transitions between close postures. However, these methods will significantly degrade the naturalness and diversity of the synthesized motion when dealing with complex and impromptu movements, e.g., dance performance and martial arts. In addition, current research lacks fine-grained control over the generated motion, which is essential for intelligent human-computer interaction and animation creation. In this paper, we propose a novel keyframe-based motion generation network based on multiple constraints, which can achieve diverse dance synthesis via learned knowledge. Specifically, the algorithm is mainly formulated based on the recurrent neural network (RNN) and the Transformer architecture. The backbone of our network is a hierarchical RNN module composed of two long short-term memory (LSTM) units, in which the first LSTM is utilized to embed the posture information of the historical frames into a latent space, and the second one is employed to predict the human posture for the next frame. Moreover, our framework contains two Transformer-based controllers, which are used to model the constraints of the root trajectory and the velocity factor respectively, so as to better utilize the temporal context of the frames and achieve fine-grained motion control. We verify the proposed approach on a dance dataset containing a wide range of contemporary dance. The results of three quantitative analyses validate the superiority of our algorithm. The video and qualitative experimental results demonstrate that the complex motion sequences generated by our algorithm can achieve diverse and smooth motion transitions between keyframes, even for long-term synthesis.

show abstract

Image Reconstruction Using Cellular Automata and Neural Networks

Loghin,

Andreica

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Neural Cellular Automata Manifold

Cited by 5 publications

References 21 publications

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

Diverse Dance Synthesis via Keyframes with Transformer Controllers

Image Reconstruction Using Cellular Automata and Neural Networks

Contact Info

Product

Resources

About