Network pruning is an important research field aiming at reducing computational costs of neural networks. Conventional approaches follow a fixed paradigm which first trains a large and redundant network, and then determines which units (e.g., channels) are less important and thus can be removed. In this work, we find that pre-training an over-parameterized model is not necessary for obtaining the target pruned structure. In fact, a fully-trained over-parameterized model will reduce the search space for the pruned structure. We empirically show that more diverse pruned structures can be directly pruned from randomly initialized weights, including potential models with better performance. Therefore, we propose a novel network pruning pipeline which allows pruning from scratch with little training overhead. In the experiments for compressing classification models on CIFAR10 and ImageNet datasets, our approach not only greatly reduces the pre-training burden of traditional pruning methods, but also achieves similar or even higher accuracy under the same computation budgets. Our results facilitate the community to rethink the effectiveness of existing techniques used for network pruning.
To address the long-standing data sparsity problem in recommender systems (RSs), cross-domain recommendation (CDR) has been proposed to leverage the relatively richer information from a richer domain to improve the recommendation performance in a sparser domain. Although CDR has been extensively studied in recent years, there is a lack of a systematic review of the existing CDR approaches. To fill this gap, in this paper, we provide a comprehensive review of existing CDR approaches, including challenges, research progress, and prospects. Specifically, we first summarize existing CDR approaches into four types, including single-target CDR, single-target multi-domain recommendation (MDR), dual-target CDR, and multi-target CDR. We then present the definitions and challenges of these CDR approaches. Next, we propose a full-view categorization and new taxonomies on these approaches and report their research progress in detail. In the end, we share several promising prospects in CDR.
Advances in non-volatile resistive switching random access memory (RRAM) have made it a promising memory technology with potential applications in low-power and embedded in-memory computing devices owing to a number of advantages such as low-energy consumption, low area cost and good scaling. There have been proposals to employ RRAM in architecting chips for neuromorphic computing and artificial neural networks where matrix-vector multiplication can be computed in the analog domain in a single timestep. However, it is challenging to employ RRAM devices in neuromorphic chips owing to the non-ideal behavior of RRAM. In this article, we propose a cycle-accurate and scalable system-level simulator that can be used to study the effects of using RRAM devices in neuromorphic computing chips. The simulator models a spatial neuromorphic chip architecture containing many neural cores with RRAM crossbars connected via a Network-on-Chip (NoC). We focus on system-level simulation and demonstrate the effectiveness of our simulator in understanding how non-linear RRAM effects such as stuck-at-faults (SAFs), write variability, and random telegraph noise (RTN) can impact an application's behavior. By using our simulator, we show that RTN and write variability can have adverse effects on an application. Nevertheless, we show that these effects can be mitigated through proper design choices and the implementation of a write-verify scheme. INTRODUCTIONNeuromorphic computing is a domain-specific computing approach that uses analog, digital, or mixed-mode integrated circuits to mimic biological architectures of the neural system, including neurons, axons, synapses, and dendrites [40]. Neurons whose inputs and outputs are spikes are used in neuromorphic computing; the resulting spike-based or spiking neural networks (SNNs) are often regarded as third-generation neural networks [39]. Special-purpose built hardware for neuromorphic computing includes the HiCANN chip [12], NeuroGrid [7], SpiNNaker [9], and IBM's TrueNorth chip [17]. SpiNNaker and TrueNorth are fully digital; HiCANN and NeuroGrid are analog or partially analog in design. In TrueNorth, 4096 neurosynaptic cores of size 256 × 256 are interconnected by an intra-chip network. Using TrueNorth to implement SNNs, Esser et al. demonstrated good accuracies in real-world application benchmarks [22].Concurrent with the developments in neuromorphic computing, advances in non-volatile resistive switching random access memory (RRAM) have made it a suitable memory technology for realizing neuromorphic computing architectures [11]. For instance, RRAM-based neuromorphic computing hardware has been proposed in [19,23,25]. Apart from advantages such as low operating power, high speed and density, memristive and RRAM-based crossbars have been proposed as energy-efficient dot-product engines. These can be used to perform matrix-vector multiplication operations efficiently in the analog domain through current sums [4,6,15]. Such approaches are suitable for low-power embedded devices targeting ne...
Point-of-Interest (POI) recommendation has been extensively studied and successfully applied in industry recently. However, most existing approaches build centralized models on the basis of collecting users’ data. Both private data and models are held by the recommender, which causes serious privacy concerns. In this article, we propose a novel Privacy preserving POI Recommendation (PriRec) framework. First, to protect data privacy, users’ private data (features and actions) are kept on their own side, e.g., Cellphone or Pad. Meanwhile, the public data that need to be accessed by all the users are kept by the recommender to reduce the storage costs of users’ devices. Those public data include: (1) static data only related to the status of POI, such as POI categories, and (2) dynamic data dependent on user-POI actions such as visited counts. The dynamic data could be sensitive, and we develop local differential privacy techniques to release such data to the public with privacy guarantees. Second, PriRec follows the representations of Factorization Machine (FM) that consists of a linear model and the feature interaction model. To protect the model privacy, the linear models are saved on the users’ side, and we propose a secure decentralized gradient descent protocol for users to learn it collaboratively. The feature interaction model is kept by the recommender since there is no privacy risk, and we adopt a secure aggregation strategy in a federated learning paradigm to learn it. To this end, PriRec keeps users’ private raw data and models in users’ own hands, and protects user privacy to a large extent. We apply PriRec in real-world datasets, and comprehensive experiments demonstrate that, compared with FM, PriRec achieves comparable or even better recommendation accuracy.
Compared with color or grayscale images, hyperspectral images deliver more informative representation of ground objects and enhance the performance of many recognition and classification applications. However, hyperspectral images are normally corrupted by various types noises, which has serious impact on the subsequent image processing tasks. In this paper, we propose a novel hyperspectral image denoising method based on tucker decomposition to model the non-local similarity across the spatial domain and global similarity along the spectral domain. In this method, 3D full band patches extracted from a hyperspectral image are grouped to form a 3rd-order tensor by utilizing the non-local similarity in a proper window size. In this way the task of image denoising is transformed into a high order tensor approximation problem, which can be solved by nonnegative tucker decomposition. Instead of traditional alternative least square based tucker decomposition, we propose a hierarchical least square based nonnegative tucker decomposition method to reduce the computational cost and improve the denoising effect. In addition, an iterative denoising strategy is adopted to achieve better denoising outcome in practice. Experimental results on three datasets show that the proposed method outperforms several state-of-the-art methods and significantly enhances the quality of the corrupted hyperspectral image.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.