In compressed sensing, one takes n < N samples of an N-dimensional vector x 0 using an n × N matrix A, obtaining undersampled measurements y = Ax 0 . For random matrices with independent standard Gaussian entries, it is known that, when x 0 is k-sparse, there is a precisely determined phase transition: for a certain region in the (k/n,n/N)-phase diagram, convex optimization min || x || 1 subject to y = Ax, x ∈ X N typically finds the sparsest solution, whereas outside that region, it typically fails. It has been shown empirically that the same property-with the same phase transition location-holds for a wide range of non-Gaussian random matrix ensembles. We report extensive experiments showing that the Gaussian phase transition also describes numerous deterministic matrices, including Spikes and Sines, Spikes and Noiselets, Paley Frames, Delsarte-Goethals Frames, Chirp Sensing Matrices, and Grassmannian Frames. Namely, for each of these deterministic matrices in turn, for a typical k-sparse object, we observe that convex optimization is successful over a region of the phase diagram that coincides with the region known for Gaussian random matrices. Our experiments considered coefficients constrained to X N for four different sets X ∈ {[0, 1], R + , R, C}, and the results establish our finding for each of the four associated phase transitions.sparse recovery | universality in random matrix theory equiangular tight frames | restricted isometry property | coherence C ompressed sensing aims to recover a sparse vector x 0 ∈ X N from indirect measurements y = Ax 0 ∈ X n with n < N, and therefore, the system of equations y = Ax 0 is underdetermined. Nevertheless, it has been shown that, under conditions on the sparsity of x 0 , by using a random measurement matrix A with Gaussian i.i.d entries and a nonlinear reconstruction technique based on convex optimization, one can, with high probability, exactly recover x 0 (1, 2). The cleanest expression of this phenomenon is visible in the large n; N asymptotic regime. We suppose that the object x 0 is k-sparse-has, at most, k nonzero entries-and consider the situation where k ∼ ρn and n ∼ δN. Fig. 1A depicts the phase diagram ðρ; δ; Þ ∈ ð0; 1Þ 2 and a curve ρ*ðδÞ separating a success phase from a failure phase. Namely, if ρ < ρ*ðδÞ, then with overwhelming probability for large N, convex optimization will recover x 0 exactly; however, if ρ > ρ*ðδÞ, then with overwhelming probability for large N convex optimization will fail. [Indeed, Fig. 1 depicts four curves ρ*ðδjXÞ of this kind for X ∈ f½0; 1; R + ; R; Cg-one for each of the different types of assumptions that we can make about the entries of x 0 ∈ X N (details below).]How special are Gaussian matrices to the above results? It was shown, first empirically in ref. 3 and recently, theoretically in ref. 4, that a wide range of random matrix ensembles exhibits precisely the same behavior, by which we mean the same phenomenon of separation into success and failure phases with the same phase boundary. Such universality, if exhib...
Recovering images from undersampled linear measurements typically leads to an ill-posed linear inverse problem, that asks for proper statistical priors. Building effective priors is however challenged by the low train and test overhead dictated by real-time tasks; and the need for retrieving visually "plausible" and physically "feasible" images with minimal hallucination. To cope with these challenges, we design a cascaded network architecture that unrolls the proximal gradient iterations by permeating benefits from generative residual networks (ResNet) to modeling the proximal operator. A mixture of pixel-wise and perceptual costs is then deployed to train proximals. The overall architecture resembles back-and-forth projection onto the intersection of feasible and plausible images. Extensive computational experiments are examined for a global task of reconstructing MR images of pediatric patients, and a more local task of superresolving CelebA faces, that are insightful to design efficient architectures. Our observations indicate that for MRI reconstruction, a recurrent ResNet with a single residual block effectively learns the proximal. This simple architecture appears to significantly outperform the alternative deep ResNet architecture by 2dB SNR, and the conventional compressed-sensing MRI by 4dB SNR with 100× faster inference. For image superresolution, our preliminary results indicate that modeling the denoising proximal demands deep ResNets.
Recovering high-resolution images from limited sensory data typically leads to a serious ill-posed inverse problem, demanding inversion algorithms that effectively capture the prior information. Learning a good inverse mapping from training data faces severe challenges, including: (i) scarcity of training data; (ii) need for plausible reconstructions that are physically feasible; (iii) need for fast reconstruction, especially in real-time applications. We develop a successful system solving all these challenges, using as basic architecture the recurrent application of proximal gradient algorithm. We learn a proximal map that works well with real images based on residual networks. Contraction of the resulting map is analyzed, and incoherence conditions are investigated that drive the convergence of the iterates. Extensive experiments are carried out under different settings: (a) reconstructing abdominal MRI of pediatric patients from highly undersampled Fourier-space data and (b) superresolving natural face images. Our key findings include: 1. a recurrent ResNet with a single residual block unrolled from an iterative algorithm yields an effective proximal which accurately reveals MR image details. 2. Our architecture significantly outperforms conventional non-recurrent deep ResNets by 2dB SNR; it is also trained much more rapidly. 3. It outperforms state-of-the-art compressed-sensing Wavelet-based methods by 4dB SNR, with 100x speedups in reconstruction time.
Modern data science research, at the cutting edge, can involve massive computational experimentation; an ambitious PhD in computational fields may conduct experiments consuming several million CPU hours. Traditional computing practices, in which researchers use laptops, PCs, or campus-resident resources with shared policies, are awkward or inadequate for experiments at the massive scale and varied scope that we now see in the most ambitious data science. On the other hand, modern cloud computing promises seemingly unlimited computational resources that can be custom configured, and seems to offer a powerful new venue for ambitious data-driven science. Exploiting the cloud fully, it seems the amount of raw experimental work that could be completed in a fixed amount of calendar time ought to expand by several orders of magnitude.As potentially powerful as cloud-based experimentation may be in the abstract, it has not yet become a standard option for researchers in many academic disciplines. The prospect of actually conducting massive computational experiments in today's cloud systems with today's standard approaches forcefully confronts the potential user with daunting challenges. The user schooled in traditional interactive personal computing likely expects that a cloud experiment will involve an intricate collection of moving parts seemingly requiring extensive monitoring and involvement. Leading considerations include: (i) the seeming complexity of today's cloud computing interface, (ii) the difficulty of executing and managing an overwhelmingly large number of computational jobs, and (iii) the difficulty of keeping track of, collating, and combining a massive collection of separate results. Starting a massive experiment 'bare-handed' seems therefore highly problematic and prone to rapid 'researcher burn out'. New software stacks are emerging that render massive cloud-based experiments relatively painless. Such stacks simplify experimentation by systematizing experiment definition, automating distribution and management of all tasks, and allowing easy harvesting of results and documentation. In this article, we discuss several painless computing stacks that abstract away the difficulties of massive experimentation, thereby allowing a proliferation of ambitious experiments for scientific discovery. * This article is based on a series of lectures given in a Stanford course Stats285 in the Fall of 2017. † Corresponding
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.