Stable low-rank tensor decomposition for compression of convolutional neural network

AH Phan, K Sobolev, K Sozykin, D Ermilov… - Computer Vision–ECCV …, 2020 - Springer
Most state-of-the-art deep neural networks are overparameterized and exhibit a high
computational cost. A straightforward approach to this problem is to replace convolutional …

Few-bit backward: Quantized gradients of activation functions for memory footprint reduction

GS Novikov, D Bershatsky, J Gusak… - International …, 2023 - proceedings.mlr.press
Memory footprint is one of the main limiting factors for large neural network training. In
backpropagation, one needs to store the input to each operation in the computational graph …

How to train unstable looped tensor network

AH Phan, K Sobolev, D Ermilov, I Vorona… - arXiv preprint arXiv …, 2022 - arxiv.org
A rising problem in the compression of Deep Neural Networks is how to reduce the number
of parameters in convolutional kernels and the complexity of these layers by low-rank tensor …

Mars: Masked automatic ranks selection in tensor decompositions

M Kodryan, D Kropotov… - … Conference on Artificial …, 2023 - proceedings.mlr.press
Tensor decomposition methods have proven effective in various applications, including
compression and acceleration of neural networks. At the same time, the problem of …

Connet: Designing a fast, efficient, and robust crowd counting model through composite compression

RMA Masilang, BJR Benedictos… - … Journal of Pattern …, 2023 - World Scientific
Counting the number of people in a specific area is crucial in maintaining proper crowd
safety and management especially in highly-congested indoor scenarios. Recent …

[PDF][PDF] Learning Low-Rank Tensor Cores with Probabilistic l0-Regularized Rank Selection for Model Compression

T Cao, L Sun, CH Nguyen, H Mamitsuka - ijcai.org
Compressing deep neural networks is of great importance for real-world applications on
resourceconstrained devices. Tensor decomposition is one promising answer that retains …