A flexible, extensible software framework for model compression based on the LC algorithm

Abstract

We propose a software framework based on the ideas of the Learning-Compression (LC) algorithm [3-5, 17, 18], that allows a user to compress a neural network or other machine learning model using different compression schemes with minimal effort. Currently, the supported compressions include pruning, quantization, low-rank methods (including automatically learning the layer ranks), and combinations of those, and the user can choose different compression types for different parts of a neural network.The LC algorithm alternates two types of steps until convergence: a learning (L) step, which trains a model on a dataset (using an algorithm such as SGD); and a compression (C) step, which compresses the model parameters (using a compression scheme such as low-rank or quantization). This decoupling of the "machine learning" aspect from the "signal compression" aspect means that changing the model or the compression type amounts to calling the corresponding subroutine in the L or C step, respectively. The library fully supports this by design, which makes it flexible and extensible. This does not come at the expense of performance: the runtime needed to compress a model is comparable to that of training the model in the first place; and the compressed model is competitive in terms of prediction accuracy and compression ratio with other algorithms (which are often specialized for specific models or compression schemes). The library is written in Python and PyTorch and available in Github.

Model compression as constrained optimization, with application to neural nets. Part V: combining compressions

“…The resulting compressed nets may also make better use of the available hardware. Our codes and models are available at https://github.com/UCMerced-ML/LC-model-compression as part of LC Toolkit [18].…”