MUSCO: Multi-Stage Compression of neural networks. The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.
References in zbMATH (referenced in 1 article )
Showing result 1 of 1.
- Hawkins, Cole; Liu, Xing; Zhang, Zheng: Towards compact neural networks via end-to-end training: a Bayesian tensor approach with automatic rank determination (2022)