Sparsemae: Sparse training meets masked autoencoders
Masked Autoencoders (MAE) and its variants have proven to be effective for pretraining
large-scale Vision Transformers (ViTs). However, small-scale models do not benefit from the …
large-scale Vision Transformers (ViTs). However, small-scale models do not benefit from the …
Efficient Model Compression Techniques with FishLeg
J McGowan, WS Lai, W Chen, H Aldridge… - arXiv preprint arXiv …, 2024 - arxiv.org
In many domains, the most successful AI models tend to be the largest, indeed often too
large to be handled by AI players with limited computational resources. To mitigate this, a …
large to be handled by AI players with limited computational resources. To mitigate this, a …
Taming transformers
J Castrillo Gutiérrez - 2023 - oa.upm.es
Since the adoption of Convolutional Neural Networks (CNNs) for vision tasks, both hardware
and software developments have rushed to make these architectures plausible on …
and software developments have rushed to make these architectures plausible on …