Generalized Sparse Regularization with Application to fMRI Brain Decoding

Abstract

Abstract. Many current medical image analysis problems involve learning thousands or even millions of model parameters from extremely few samples. Employing sparse models provides an effective means for handling the curse of dimensionality, but other propitious properties beyond sparsity are typically not modeled. In this paper, we propose a simple approach, generalized sparse regularization (GSR), for incorporating domain-specific knowledge into a wide range of sparse linear models, such as the LASSO and group LASSO regression models. We demonstrate the power of GSR by building anatomically-informed sparse classifiers that additionally model the intrinsic spatiotemporal characteristics of brain activity for fMRI classification. We validate on real data and show how prior-informed sparse classifiers outperform standard classifiers, such as SVM and a number of sparse linear classifiers, both in terms of prediction accuracy and result interpretability. Our results illustrate the addedvalue in facilitating flexible integration of prior knowledge beyond sparsity in large-scale model learning problems.Keywords: brain decoding, fMRI classification, prior-informed learning, sparse optimization, spatiotemporal regularization IntroductionRecent years witnessed a surging interest in exploiting sparsity [1-9] to handle the ever-increasing scale and complexity of current medical image analysis problems [10,11]. Oftentimes, one is faced with exceedingly more predictors than samples. Under such ill-conditioned settings, incorporating sparsity into model learning proved to be of enormous benefits. In particular, enforcing sparsity enables model parameters associated with irrelevant predictors to be implicitly removed, i.e. shrunk to exactly zero [1], which reduces overfitting thus enhances model generalizability. Learning parsimonious models by imposing sparsity also simplifies result interpretation [1], which is of utmost importance in most medical studies. Since the advent of the least absolute shrinkage and selection operator (LASSO) regression model [1], where Tibshirani showed that penalizing the l 1 norm induces sparsity in the regression coefficients, numerous powerful variants were subsequently proposed [2][3][4][5][6][7][8][9]. Zou and Hastie, for instance, proposed the elastic net penalty [2], which retains the sparse property of LASSO but additionally encourages correlated

Vehicle Detection in Aerial Traffic Monitoring

“…Following hidden layers are PCA. When training the first monolayer autoencoder KL regularization and weight decay regularization are used (Ng, 2011). When training the next monolayer autoencoders only KL is used.…”

A New Deep Learning Approach for Anomaly Base IDS using Memetic Classifier

“…Auto-encoder is based on the concept of Sparse coding [11]. AE can be considered as a discriminative DNN in which the target output would be similar to the input, and the number of hidden layer nodes is lower than input.…”

Feature Descriptor by Convolution and Pooling Autoencoders

“…A feature vector can be computed as h= f θ (x) from an input x with a form f θ (x)=(b+W e x), where  is the logistic sigmoid function, serving as the activation function (Ng, 2011):…”

“…To prevent overfitting, a regularization term is also added to the Euclidean norm of the difference. The overall cost function is (Ng, 2011): The weight parameter λ controls the relative importance of the regularization term compared to the reconstruction error term. One can learn the parameters θ={W e , b, W d , d} of the encoder and the decoder simultaneously by minimizing the cost function J().…”