Model compression and acceleration for deep neural networks: The principles, progress, and challenges

Y Cheng, D Wang, P Zhou… - IEEE Signal Processing …, 2018 - ieeexplore.ieee.org
In recent years, deep neural networks (DNNs) have received increased attention, have been
applied to different applications, and achieved dramatic accuracy improvements in many …

A survey of model compression and acceleration for deep neural networks

Y Cheng, D Wang, P Zhou, T Zhang - arXiv preprint arXiv:1710.09282, 2017 - arxiv.org
Deep neural networks (DNNs) have recently achieved great success in many visual
recognition tasks. However, existing deep neural network models are computationally …

Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions

A Cichocki, N Lee, I Oseledets, AH Phan… - … and Trends® in …, 2016 - nowpublishers.com
Modern applications in engineering and data science are increasingly based on
multidimensional data of exceedingly high volume, variety, and structural richness …

Low-rank tensor networks for dimensionality reduction and large-scale optimization problems: Perspectives and challenges part 1

A Cichocki, N Lee, IV Oseledets, AH Phan… - arXiv preprint arXiv …, 2016 - arxiv.org
Machine learning and data mining algorithms are becoming increasingly important in
analyzing large volume, multi-relational and multi--modal datasets, which are often …

Structured transforms for small-footprint deep learning

V Sindhwani, T Sainath… - Advances in Neural …, 2015 - proceedings.neurips.cc
We consider the task of building compact deep learning pipelines suitable for deploymenton
storage and power constrained mobile devices. We propose a uni-fied framework to learn a …

A quantum-inspired approach to exploit turbulence structures

N Gourianov, M Lubasch, S Dolgov… - Nature Computational …, 2022 - nature.com
Understanding turbulence is key to our comprehension of many natural and technological
flow processes. At the heart of this phenomenon lies its intricate multiscale nature …

Online ensemble model compression using knowledge distillation

D Walawalkar, Z Shen, M Savvides - … , Glasgow, UK, August 23–28, 2020 …, 2020 - Springer
This paper presents a novel knowledge distillation based model compression framework
consisting of a student ensemble. It enables distillation of simultaneously learnt ensemble …

Tensor numerical methods for multidimensional PDEs: theoretical analysis and initial applications

BN Khoromskij - ESAIM: Proceedings and Surveys, 2015 - esaim-proc.org
We present a brief survey on the modern tensor numerical methods for multidimensional
stationary and time-dependent partial differential equations (PDEs). The guiding principle of …

Cross tensor approximation methods for compression and dimensionality reduction

S Ahmadi-Asl, CF Caiafa, A Cichocki, AH Phan… - IEEE …, 2021 - ieeexplore.ieee.org
Cross Tensor Approximation (CTA) is a generalization of Cross/skeleton matrix and CUR
Matrix Approximation (CMA) and is a suitable tool for fast low-rank tensor approximation. It …

Calculating vibrational spectra of molecules using tensor train decomposition

M Rakhuba, I Oseledets - The Journal of chemical physics, 2016 - pubs.aip.org
We propose a new algorithm for calculation of vibrational spectra of molecules using tensor
train decomposition. Under the assumption that eigenfunctions lie on a low-parametric …