No abstract

Abstract. We develop a probabilistic interpretation of non-linear component extraction in neural networks that activate their hidden units according to a softmaxlike mechanism. On the basis of a generative model that combines hidden causes using the max-function, we show how the extraction of input components in such networks can be interpreted as maximum likelihood parameter optimization. A simple and neurally plausible Hebbian Δ-rule is derived. For approximatelyoptimal learning, the activity of the hidden neural units is described by a generalized softmax function and the classical softmax is recovered for very sparse input. We use the bars benchmark test to numerically verify our analytical results and to show competitiveness of the derived learning algorithms.

No abstract

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.

Feedforward inhibition and synaptic scaling are important adaptive processes that control the total input a neuron can receive from its afferents. While often studied in isolation, the two have been reported to co-occur in various brain regions. The functional implications of their interactions remain unclear, however. Based on a probabilistic modeling approach, we show here that fast feedforward inhibition and synaptic scaling interact synergistically during unsupervised learning. In technical terms, we model the input to a neural circuit using a normalized mixture model with Poisson noise. We demonstrate analytically and numerically that, in the presence of lateral inhibition introducing competition between different neurons, Hebbian plasticity and synaptic scaling approximate the optimal maximum likelihood solutions for this model. Our results suggest that, beyond its conventional use as a mechanism to remove undesired pattern variations, input normalization can make typical neural interaction and learning rules optimal on the stimulus subspace defined through feedforward inhibition. Furthermore, learning within this subspace is more efficient in practice, as it helps avoid locally optimal solutions. Our results suggest a close connection between feedforward inhibition and synaptic scaling which may have important functional implications for general cortical processing.

We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated by neuroanatomical and neurophysiological findings the utilized dynamics is based on a simple model of a spiking neuron with refractory period, fixed random excitatory interconnection within minicolumns, and instantaneous inhibition within one macrocolumn. A stability analysis of the system's dynamical equations shows that minicolumns can act as monolithic functional units for purposes of critical, fast decisions and learning. Oscillating inhibition (in the gamma frequency range) leads to a phase-coupled population rate code and high sensitivity to small imbalances in minicolumn inputs. Minicolumns are shown to be able to organize their collective inputs without supervision by Hebbian plasticity into selective receptive field shapes, thereby becoming classifiers for input patterns. Using the bars test, we critically compare our system's performance with that of others and demonstrate its ability for distributed neural coding.

Personalized medicine is a modern healthcare approach where information on each person's unique clinical constitution is exploited to realize early disease intervention based on more informed medical decisions. The application of diagnostic tools in combination with measurement evaluation that can be performed in a reliable and automated fashion plays a key role in this context. As the progression of various cancer diseases and the effectiveness of their treatments are related to a varying number of tumor cells that circulate in blood, the determination of their extremely low numbers by liquid biopsy is a decisive prognostic marker. To detect and enumerate circulating tumor cells (CTCs) in a reliable and automated fashion, we apply methods from machine learning using a naive Bayesian classifier (NBC) based on a probabilistic generative mixture model. Cells are collected with a functionalized medical wire and are stained for fluorescence microscopy so that their color signature can be used for classification through the construction of Red-Green-Blue (RGB) color histograms. Exploiting the information on the fluorescence signature of CTCs by the NBC does not only allow going beyond previous approaches but also provides a method of unsupervised learning that is required for unlabeled training data. A quantitative comparison with a state-of-the-art support vector machine, which requires labeled data, demonstrates the competitiveness of the NBC method. (5-7), much emphasis has been put on techniques that effectively collect CTCs from peripheral blood. Currently, the majority of studies use anti-epithelial-cell-adhesionmolecule (EpCAM) antibody-coated isolation systems (8-10), whereas other types of immunomagnetic devices (11-13), density gradient centrifugation (14) or membrane filtration (15,16), are also used for CTC enrichment. The subsequent CTC detection is realized by either immunocytological staining or polymerase chain reaction (17). Due to the extraordinary rarity of CTCs in patients, which have a typical ratio of one CTC per 10 8 blood cells (18), classical flow cytometry has proven unreliable and methods using intravital multiphoton flow cytometry and photoacoustic flowmetry are currently being developed (17,(19)(20)(21)(22). Although these methods have already successfully identified CTCs in control blood specimens spiked with cancer cell lines, they are not yet ready for use in detection of CTC in patients with cancer.The data of this study were generated by a technology that can collect CTCs in vivo using a functionalized and structured medical wire (FSMW) (23). Medical guidewires are widely used for angiography and insertion of venous catheters (24)

We show that k-means (Lloyd's algorithm) is obtained as a special case when truncated variational EM approximations are applied to Gaussian Mixture Models (GMM) with isotropic Gaussians. In contrast to the standard way to relate k-means and GMMs, the provided derivation shows that it is not required to consider Gaussians with small variances or the limit case of zero variances. There are a number of consequences that directly follow from our approach: (A) k-means can be shown to increase a free energy associated with truncated distributions and this free energy can directly be reformulated in terms of the k-means objective; (B) k-means generalizations can directly be derived by considering the 2nd closest, 3rd closest etc. cluster in addition to just the closest one; and (C) the embedding of k-means into a free energy framework allows for theoretical interpretations of other k-means generalizations in the literature. In general, truncated variational EM provides a natural and rigorous quantitative link between k-means-like clustering and GMM clustering algorithms which may be very relevant for future theoretical and empirical studies.