Combating Adversarial Inputs Using a Predictive-Estimator Network

Orchard, Jeff; Castricato, Louis

doi:10.1007/978-3-319-70096-0_13

Cited by 4 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I would speculate that the recurrent loop used by the denoising sampler can be used to make the input images more prototypical which reduces the impact of the low probability events modeled by the PWA. This result is not that surprising as feedback loops have already been shown to improve adversarial robustness (Orchard & Castricato, 2017). This completes my discussion of the inductive biases of the PWA.…”

Section: Discussionsupporting

confidence: 66%

Asymptotic Graph Neural Networks Improve Adversarial Robustness in Machine Vision

Vertolli¹

View full text Add to dashboard Cite

Section: Discussionsupporting

confidence: 66%

Asymptotic Graph Neural Networks Improve Adversarial Robustness in Machine Vision

Vertolli¹

View full text Add to dashboard Cite

“…For example, perception and inference could be the result of running the network in feed-forward and feed-back directions simultaneously, like in the wake-sleep approach [25]. Moreover, predictive estimator networks might be a natural implementation for such feedback networks [5], [26], [27]. Continuing in this vein is left for future work and these experiments are ongoing.…”

Section: Discussionmentioning

confidence: 99%

Style Memory: Making a Classifier Network Generative

Wiyatno

Orchard

2018

2018 IEEE 17th International Conference on Cognitive Informatics &Amp; Cognitive Computing (ICCI*CC)

Self Cite

View full text Add to dashboard Cite

Deep networks have shown great performance in classification tasks. However, the parameters learned by the classifier networks usually discard stylistic information of the input, in favour of information strictly relevant to classification. We introduce a network that has the capacity to do both classification and reconstruction by adding a "style memory" to the output layer of the network. We also show how to train such a neural network as a deep multi-layer autoencoder, jointly minimizing both classification and reconstruction losses. The generative capacity of our network demonstrates that the combination of style-memory neurons with the classifier neurons yield good reconstructions of the inputs when the classification is correct. We further investigate the nature of the style memory, and how it relates to composing digits and letters. Finally, we propose that this architecture enables the bidirectional flow of information used in predictive coding, and that such bidirectional networks can help mitigate against being fooled by ambiguous or adversarial input.

show abstract

“…Generative networks might also offer improved robustness to noisy, ambiguous, or adversarial inputs (Orchard & Castricato, 2017) and may play a role in perceiving Gestalt forms (Bartels, 2014).…”

Section: Introductionmentioning

confidence: 99%

A Predictive-Coding Network That Is Both Discriminative and Generative

Sun

Orchard

2020

Neural Computation

Self Cite

View full text Add to dashboard Cite

Predictive coding (PC) networks are a biologically interesting class of neural networks. Their layered hierarchy mimics the reciprocal connectivity pattern observed in the mammalian cortex, and they can be trained using local learning rules that approximate backpropagation (Bogacz, 2017). However, despite having feedback connections that enable information to flow down the network hierarchy, discriminative PC networks are not typically generative. Clamping the output class and running the network to equilibrium yields an input sample that usually does not resemble the training input. This letter studies this phenomenon and proposes a simple solution that promotes the generation of input samples that resemble the training inputs. Simple decay, a technique already in wide use in neural networks, pushes the PC network toward a unique minimum two-norm solution, and that unique solution provably (for linear networks) matches the training inputs. The method also vastly improves the samples generated for nonlinear networks, as we demonstrate on MNIST.

show abstract

Combating Adversarial Inputs Using a Predictive-Estimator Network

Cited by 4 publications

References 4 publications

Asymptotic Graph Neural Networks Improve Adversarial Robustness in Machine Vision

Asymptotic Graph Neural Networks Improve Adversarial Robustness in Machine Vision

Style Memory: Making a Classifier Network Generative

A Predictive-Coding Network That Is Both Discriminative and Generative

Contact Info

Product

Resources

About