SqueezeFit: Label-Aware Dimensionality Reduction by Semidefinite Programming

McWhirter, Culver; Mixon, Dustin G.; Villar, Soledad

doi:10.1109/tit.2019.2962681

Cited by 7 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The drop in performance might reflect the fact that MCA is oblivious to the data labels, suggesting a label-aware alternative (cf. PCA vs. SqueezeFit [15]). The performance drop might also reflect our choice of affine linear maps and Euclidean distances, suggesting alternatives involving non-linear maps and other distances.…”

Section: Discussionmentioning

confidence: 99%

Matching Component Analysis for Transfer Learning

Clum¹,

Mixon²,

Scarnati³

2020

SIAM Journal on Mathematics of Data Science

Self Cite

View full text Add to dashboard Cite

We introduce a new Procrustes-type method called matching component analysis to isolate components in data for transfer learning. Our theoretical results describe the sample complexity of this method, and we demonstrate through numerical experiments that our approach is indeed well suited for transfer learning. IntroductionMany state-of-the-art classification algorithms require a large training set that is statistically similar to the test set. For example, deep learning-based approaches require a large number of representative samples in order to find near-optimal network weights and biases [4,9]. Similarly, template-based approaches require large dictionaries of training images so that each test image can be represented by an element of the dictionary [21,26,17,6]. For each technique, if test images cannot be represented in a feature space that has been determined from the training set, then classification accuracy is poor.In applications such as synthetic aperture radar (SAR) automatic target recognition (ATR), it is infeasible to collect the volume of data necessary to naively train high-accuracy classification networks. Additionally, due to varying operating conditions, the features measured in SAR imagery are different from those extracted from electro-optical (EO) imagery [13]. As such, off-the-shelf networks that have been pre-trained on the popular EObased ImageNet [3] or CIFAR-10 [8] datasets are insufficient for performing accurate ATR tasks in different imaging domains. In fact, recent work has demonstrated that pre-trained networks fail to effectively generalize to random perturbations on test sets [19,18]. To build more representative training sets, additional data are often generated using modeling and simulation software. However, due to various model errors, simulated data often misrepresent the real-world scattering observed in measured imagery. Thus, even though it is possible to augment training sets with a large amount of simulated data, the inherent differences in sensor modalities and data representations make modifying classification networks a non-trivial task [20].In this paper, we introduce matching component analysis (MCA) to help remedy this situation. Given a small number of images from the training domain and matching *

show abstract

Section: Discussionmentioning

confidence: 99%

Matching Component Analysis for Transfer Learning

Clum¹,

Mixon²,

Scarnati³

2020

SIAM Journal on Mathematics of Data Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Setup. We model the marker selection problem as a label-aware dimension reduction method inspired by compressive classification and largest margin nearest neighbor algorithms [McWhirter et al, 2018]. One such method, SqueezeFit, aims to find a projection to the lowest dimensional subspace for which samples with different labels remain farther apart than samples with the same label.…”

Section: Discussionmentioning

confidence: 99%

“…This parameter reflects a fundamental tension in compressive classification: ∆ should be large so as to enable sufficient separation of samples with different labels in the low dimensional space, and simultaneously the projected space rank Π should be of low dimension so that this projection effectively reduces the dimension of the sample. To address the intractability of the optimization in (1), a convex relaxation technique is used [McWhirter et al, 2018]:…”

Section: Discussionmentioning

confidence: 99%

“…Briefly, scGeneFit works as follows. Given samples (cells) in a high-dimensional feature space (genes), and corresponding categorical sample labels (e.g., cell type, cell state), label-aware compression methods [McWhirter et al, 2018] find a projection to a low dimensional subspace (space of markers, where the dimension is specified) where samples with different labels remain separated when projected into that space. To ensure that the low-dimensional results enable marker selection, scGeneFit additionally constrains the projection such that each of the subspace dimensions must be aligned with a coordinate axis in the original space.…”

mentioning

confidence: 99%

“…scGeneFit identifies markers associated with a flat partition of cell type labels when applied to simulated data and to scRNA-seq data. Panel A: Proof of concept (cf [McWhirter et al, 2018]…”

mentioning

confidence: 99%

See 2 more Smart Citations

Optimal marker gene selection for cell type discrimination in single cell analyses

Dumitrascu

Villar

Mixon

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Single-cell technologies characterize complex cell populations across multiple data modalities at un-precedented scale and resolution. Multi-omic data for single cell gene expression,in situhybridization, or single cell chromatin states are increasingly available across diverse tissue types. When isolating specific cell types from a sample of disassociated cells or performingin situsequencing in collections of heterogeneous cells, one challenging task is to select a small set of informative markers to identify and differentiate specific cell types or cell states as precisely as possible. Given single cell RNA-seq data and a set of cellular labels to discriminate, scGene-Fit selects gene transcript markers that jointly optimize cell label recovery using label-aware compressive classification methods, resulting in a substantially more robust and less redundant set of markers than existing methods. When applied to a data set given a hierarchy of cell type labels, the markers found by our method enable the recovery of the label hierarchy through a computationally efficient and principled optimization.

show abstract