Neural Network Distiller: A Python Package For DNN Compression Research

Zmora, Neta; Jacob, Guy; Zlotnik, Lev; Elharar, Bar; Novik, Gal

doi:10.48550/arxiv.1910.12232

Cited by 8 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The diversity compression mechanisms and limited support by deep learning frameworks led to development of specialized software libraries such as Distiller [45] and NCCF [21]. Distiller and NCCF gather multiple compression schemes and corresponding training algorithms into a single framework, and make it easier to apply to new models.…”

Section: Compression Frameworkmentioning

confidence: 99%

A flexible, extensible software framework for model compression based on the LC algorithm

Idelbayev¹,

Carreira-Perpiñán²

2020

Preprint

View full text Add to dashboard Cite

We propose a software framework based on the ideas of the Learning-Compression (LC) algorithm [3-5, 17, 18], that allows a user to compress a neural network or other machine learning model using different compression schemes with minimal effort. Currently, the supported compressions include pruning, quantization, low-rank methods (including automatically learning the layer ranks), and combinations of those, and the user can choose different compression types for different parts of a neural network.The LC algorithm alternates two types of steps until convergence: a learning (L) step, which trains a model on a dataset (using an algorithm such as SGD); and a compression (C) step, which compresses the model parameters (using a compression scheme such as low-rank or quantization). This decoupling of the "machine learning" aspect from the "signal compression" aspect means that changing the model or the compression type amounts to calling the corresponding subroutine in the L or C step, respectively. The library fully supports this by design, which makes it flexible and extensible. This does not come at the expense of performance: the runtime needed to compress a model is comparable to that of training the model in the first place; and the compressed model is competitive in terms of prediction accuracy and compression ratio with other algorithms (which are often specialized for specific models or compression schemes). The library is written in Python and PyTorch and available in Github.

show abstract

Section: Compression Frameworkmentioning

confidence: 99%

A flexible, extensible software framework for model compression based on the LC algorithm

Idelbayev¹,

Carreira-Perpiñán²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…size by quantizing model parameters and pruning redundant neurons, and many of such methods are covered by Distiller [53], an open-source library for model compression.…”

Section: Frameworkmentioning

confidence: 99%

“…However, such frameworks are usually not either well generalized or maintained to be built on. Besides, Distiller [53] supports only one method for knowledge distillation, and Catalyst [16] is a framework built on PyTorch with a focus on reproducibility of deep learning research. To support various deep learning methods, these frameworks are well generalized, yet require users to hardcode (reimplement) critical modules such as models and datasets, even if the implementations are publicly available in popular libraries, to design complex knowledge distillation experiments.…”

Section: Frameworkmentioning

confidence: 99%

torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

Matsubara

2020

Preprint

View full text Add to dashboard Cite

While knowledge distillation (transfer) has been attracting attentions from the research community, the recent development in the fields has heightened the need for reproducible studies and highly generalized frameworks to lower barriers to such high-quality, reproducible deep learning research. Several researchers voluntarily published frameworks used in their knowledge distillation studies to help other interested researchers reproduce their original work. Such frameworks, however, are usually neither well generalized nor maintained, thus researchers are still required to write a lot of code to refactor/build on the frameworks for introducing new methods, models, datasets and designing experiments. In this paper, we present our developed open-source framework built on PyTorch and dedicated for knowledge distillation studies. The framework is designed to enable users to design experiments by a declarative PyYAML configuration file, and helps researchers complete the recently proposed ML Code Completeness Checklist. Using the developed framework, we demonstrate its various efficient training strategies, and implement a variety of knowledge distillation methods. We also reproduce some of their original experimental results on the ImageNet and COCO datasets presented at major machine learning conferences such as ICLR, NeurIPS, CVPR and ECCV, including recent state-of-the-art methods. All the source code, configurations, log files and the trained model weights are publicly available at https://github.com/yoshitomo-matsubara/torchdistill.

show abstract

“…For instance, [31] sequentially prunes and retrains on a per layer basis, while works such as [37] have to add many auxiliary layers on top of the chosen architecture in order to create and train their early exit classifiers. Distiller [41] and Mayo [39] are two state-of-the-art open-source frameworks that allow for experimentation with such pruning techniques. Mayo focuses on automating search for hyperpameters related to pruning, while Distiller focuses on implementing a wide variety of pruning techniques discussed above.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Based on this information, the PDC automatically calculates all the dependent layers and communicates this information to the pruning stage so layers can be pruned in dependent groups if necessary. This automation makes ADaPT very easy to use as tools such as Distiller [41] require the user to manually identify each dependent convolution in the entire network, which for larger networks can be very tedious to list.…”

Section: Pruning Dependency Calculation (Pdc)mentioning

confidence: 99%

Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge

Rajagopal¹,

Bouganis²

2020

Preprint

View full text Add to dashboard Cite

In today's world, a vast amount of data is being generated by edge devices that can be used as valuable training data to improve the performance of machine learning algorithms in terms of the achieved accuracy or to reduce the compute requirements of the model. However, due to user data privacy concerns as well as storage and communication bandwidth limitations, this data cannot be moved from the device to the data centre for further improvement of the model and subsequent deployment. As such there is a need for increased edge intelligence, where the deployed models can be fine-tuned on the edge, leading to improved accuracy and/or reducing the model's workload as well as its memory and power footprint. In the case of Convolutional Neural Networks (CNNs), both the weights of the network as well as its topology can be tuned to adapt to the data that it processes. This paper provides a first step towards enabling CNN finetuning on an edge device based on structured pruning. It explores the performance gains and costs of doing so and presents an extensible open-source framework that allows the deployment of such approaches on a wide range of network architectures and devices. The results show that on average, data-aware pruning with retraining can provide 10.2pp increased accuracy over a wide range of subsets, networks and pruning levels with a maximum improvement of 42.0pp over pruning and retraining in a manner agnostic to the data being processed by the network.

show abstract

Neural Network Distiller: A Python Package For DNN Compression Research

Cited by 8 publications

References 14 publications

A flexible, extensible software framework for model compression based on the LC algorithm

A flexible, extensible software framework for model compression based on the LC algorithm

torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge

Contact Info

Product

Resources

About