Toward a neuro-inspired creative decoder

Abstract

<jats:p>Creativity, a process that generates novel and meaningful ideas, involves increased association between task-positive (control) and task-negative (default) networks in the human brain. Inspired by this seminal finding, in this study we propose a creative decoder within a deep generative framework, which involves direct modulation of the neuronal activation pattern after sampling from the learned latent space. The proposed approach is fully unsupervised and can be used off- the-shelf. Several novelty metrics and human evaluation were used to evaluate the creative capacity of the deep decoder. Our experiments on different image datasets (MNIST, FMNIST, MNIST+FMNIST, WikiArt and CelebA) reveal that atypical co-activation of highly activated and weakly activated neurons in a deep decoder promotes generation of novel and meaningful artifacts.</jats:p>

Towards Creativity Characterization of Generative Models via Group-Based Subset Scanning

“…We test this hypothesis through group-based subset scanning over the activation space that encodes groups of artifacts that may appear anomalous when analyzed together. Given that the proposed approach is model agnostic, we test it under two generative models, a creative VAE Decoder and a Creative Generator variant of ArtGAN architectures [Das et al, 2020;Tan et al, 2019]. We scan both the pixel and activation space.…”

“…We scan both the pixel and activation space. We also validate the proposed approach across multiple datasets, including images from MNIST, Fashion-MNIST (FMNIST) [Xiao et al, 2017], Combo [Das et al, 2020], which is a combination of the previous two datasets, and WikiArt (with ArtGAN) [Tan et al, 2019].…”

“…Where CD and G is a generative neural network capable of producing creative outputs [Das et al, 2020]. Let X S ⊆ X and O S ⊆ O, we then define the subsets S under consideration to be S = X S × O S .…”

“…For human evaluation under FMNIST, MNIST, and Combo datasets, we use the labeled artifacts from [Das et al, 2020] to test the performance of our proposed method. Nine evaluators did the human annotation over a pool of 500 samples per dataset (we used agreement amongst > 3 annotators as consensus), generated from either the creative generative process or regular decoding.…”

“…In short, this work identifies which, of the exponentiallymany, subset of samples has higher-than-expected activations at which, of the subset of nodes in a hidden layer of a generative model. For that purpose, we present experiments with a "Creative" variant [Das et al, 2020] of a VAE decoder and of an ArtGAN generator [Tan et al, 2019]. Furthermore, we provide a quantitative, as well as a qualitative, assessment under the WikiArt dataset and the creative samples guided selection process to evaluate the possibility of using group-based scanning as a proxy for the novelty component in creativity perception.…”