Representation and separation of signals using nonlinear PCA type learning

Karhunen, Juha; Joutsensalo, J.

doi:10.1016/0893-6080(94)90060-4

Cited by 328 publications

(151 citation statements)

References 15 publications

Supporting

Mentioning

149

Contrasting

Unclassified

Order By: Relevance

“…This enables the use of non-linear generative models, such as the Helmholtz machine for binary stochastic systems and non-linear PCA for parametric deterministic models (e.g. Dong & McAvoy, 1996;Friston et al, 2000;Karhunen & Joutsensalo, 1994;Kramer, 1991;Taleb & Jutten, 1997). The latter schemes typically employ a 'bottleneck' architecture that forces the inputs through a small number of nodes.…”

Section: Non-invertible Modelsmentioning

confidence: 99%

Learning and inference in the brain

2003

View full text Add to dashboard Cite

Section: Non-invertible Modelsmentioning

confidence: 99%

Learning and inference in the brain

2003

View full text Add to dashboard Cite

“…This enables the use of nonlinear generative models, such as nonlinear PCA (e.g. Kramer, 1991;Karhunen and Joutsensalo, 1994;Dong and McAvoy, 1996;Taleb and Jutten, 1997). These schemes typically employ a 'bottleneck' architecture that forces the inputs through a small number of nodes.…”

Section: Information Theorymentioning

confidence: 99%

Functional integration and inference in the brain

Friston

2002

Progress in Neurobiology

271

201

View full text Add to dashboard Cite

Self-supervised models of how the brain represents and categorises the causes of its sensory input can be divided into two classes: those that minimise the mutual information (i.e. redundancy) among evoked responses and those that minimise the prediction error. Although these models have similar goals, the way they are attained, and the functional architectures employed, can be fundamentally different. This review describes the two classes of models and their implications for the functional anatomy of sensory cortical hierarchies in the brain. We then consider how empirical evidence can be used to disambiguate between architectures that are sufficient for perceptual learning and synthesis.Most models of representational learning require prior assumptions about the distribution of sensory causes. Using the notion of empirical Bayes, we show that these assumptions are not necessary and that priors can be learned in a hierarchical context. Furthermore, we try to show that learning can be implemented in a biologically plausible way. The main point made in this review is that backward connections, mediating internal or generative models of how sensory inputs are caused, are essential if the process generating inputs cannot be inverted. Because these processes are dynamical in nature, sensory inputs correspond to a non-invertible nonlinear convolution of causes. This enforces an explicit parameterisation of generative models (i.e. backward connections) to enable approximate recognition and suggests that feedforward architectures, on their own, are not sufficient. Moreover, nonlinearities in generative models, that induce a dependence on backward connections, require these connections to be modulatory; so that estimated causes in higher cortical levels can interact to predict responses in lower levels. This is important in relation to functional asymmetries in forward and backward connections that have been demonstrated empirically.To ascertain whether backward influences are expressed functionally requires measurements of functional integration among brain systems. This review summarises approaches to integration in terms of effective connectivity and proceeds to address the question posed by the theoretical considerations above. In short, it will be shown that functional neuroimaging can be used to test for interactions between bottom-up and top-down inputs to an area. The conclusion of these studies points toward the prevalence of top-down influences and the plausibility of generative models of sensory brain function.

show abstract

“…-Preprocessing: we first calculate and subtract the average pattern to obtain a zero mean process (Karhunen & Joutsensalo 1994. -Neural computing: the fundamental learning parameters are: i) the initial weight matrix; ii) the number of input neurons L, the number of output neurons p, which is the number of principal eigenvectors that we need, and therefore is equal to twice the number of signal periodicities (for real signals); iii) α, the nonlinear learning function parameter; iv) the learning rate µ.…”

Section: Autocorrelation Matrix Based Analysismentioning

confidence: 99%

A multifrequency analysis of radio variability of blazars

Ciaramella

Bongardo

Aller

et al. 2004

A&A

View full text Add to dashboard Cite

Abstract.We have carried out a multifrequency analysis of the radio variability of blazars, exploiting the data obtained during the extensive monitoring programs carried out at the University of Michigan Radio Astronomy Observatory (UMRAO, at 4.8, 8, and 14.5 GHz) and at the Metsähovi Radio Observatory (22 and 37 GHz). Two different techniques detect, in the Metsähovi light curves, evidence of periodicity at both frequencies for 5 sources (0224 + 671, 0945 + 408, 1226 + 023, 2200 + 420, and 2251 + 158). For the last three sources, consistent periods are found also at the three UMRAO frequencies and the Scargle (1982) method yields an extremely low false-alarm probability. On the other hand, the 22 and 37 GHz periodicities of 0224+671 and 0945 + 408 (which were less extensively monitored at Metsähovi and for which we get a significant false-alarm probability) are not confirmed by the UMRAO database, where some indications of ill-defined periods of about a factor of two longer are retrieved. We have also investigated the variability index, the structure function, and the distribution of intensity variations of the most extensively monitored sources. We find a statistically significant difference in the distribution of the variability index for BL Lac objects compared to flat-spectrum radio quasars (FSRQs), in the sense that the former objects are more variable. For both populations the variability index steadily increases with increasing frequency. The distribution of intensity variations also broadens with increasing frequency, and approaches a log-normal shape at the highest frequencies. We find that variability enhances by 20-30% the high frequency counts of extragalactic radio-sources at bright flux densities, such as those of the WMAP and P surveys. In all objects with detected periodicity we find evidence for the existence of impulsive signals superimposed on the periodic component.

show abstract

Representation and separation of signals using nonlinear PCA type learning

Cited by 328 publications

References 15 publications

Learning and inference in the brain

Learning and inference in the brain

Functional integration and inference in the brain

A multifrequency analysis of radio variability of blazars

Contact Info

Product

Resources

About