[HTML][HTML] To compress or not to compress—self-supervised learning and information theory: A review
R Shwartz Ziv, Y LeCun - Entropy, 2024 - mdpi.com
Deep neural networks excel in supervised learning tasks but are constrained by the need for
extensive labeled data. Self-supervised learning emerges as a promising alternative …
extensive labeled data. Self-supervised learning emerges as a promising alternative …
Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery
Unsupervised pre-training methods for large vision models have shown to enhance
performance on downstream supervised tasks. Developing similar techniques for satellite …
performance on downstream supervised tasks. Developing similar techniques for satellite …
An introduction to neural data compression
Neural compression is the application of neural networks and other machine learning
methods to data compression. Recent advances in statistical machine learning have opened …
methods to data compression. Recent advances in statistical machine learning have opened …
Improving self-supervised learning by characterizing idealized representations
Despite the empirical successes of self-supervised learning (SSL) methods, it is unclear
what characteristics of their representations lead to high downstream accuracies. In this …
what characteristics of their representations lead to high downstream accuracies. In this …
Task-oriented communication for multidevice cooperative edge inference
This paper investigates task-oriented communication for multi-device cooperative edge
inference, where a group of distributed low-end edge devices transmit the extracted features …
inference, where a group of distributed low-end edge devices transmit the extracted features …
Compute-efficient deep learning: Algorithmic trends and opportunities
BR Bartoldson, B Kailkhura, D Blalock - Journal of Machine Learning …, 2023 - jmlr.org
Although deep learning has made great progress in recent years, the exploding economic
and environmental costs of training neural networks are becoming unsustainable. To …
and environmental costs of training neural networks are becoming unsustainable. To …
Supervised compression for resource-constrained edge computing systems
There has been much interest in deploying deep learning algorithms on low-powered
devices, including smartphones, drones, and medical sensors. However, full-scale deep …
devices, including smartphones, drones, and medical sensors. However, full-scale deep …
Optimal representations for covariate shift
Machine learning systems often experience a distribution shift between training and testing.
In this paper, we introduce a simple variational objective whose optima are exactly the set of …
In this paper, we introduce a simple variational objective whose optima are exactly the set of …
Is a caption worth a thousand images? a controlled study for representation learning
The development of CLIP [Radford et al., 2021] has sparked a debate on whether language
supervision can result in vision models with more transferable representations than …
supervision can result in vision models with more transferable representations than …
Compressive visual representations
Learning effective visual representations that generalize well without human supervision is a
fundamental problem in order to apply Machine Learning to a wide variety of tasks …
fundamental problem in order to apply Machine Learning to a wide variety of tasks …