Llama 2: Early Adopters' Utilization of Meta's New Open-Source Pretrained Model

KI Roumeliotis, ND Tselikas, DK Nasiopoulos - 2023 - preprints.org
The rapidly evolving field of artificial intelligence (AI) continues to witness the introduction of
innovative open-source pre-trained models, fostering advancements in various applications …

Accurate and structured pruning for efficient automatic speech recognition

H Jiang, LL Zhang, Y Li, Y Wu, S Cao, T Cao… - arXiv preprint arXiv …, 2023 - arxiv.org
Automatic Speech Recognition (ASR) has seen remarkable advancements with deep neural
networks, such as Transformer and Conformer. However, these models typically have large …

Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation

S Sanyal, RV Babu - … of the IEEE/CVF Conference on …, 2023 - openaccess.thecvf.com
In this paper, we propose to develop a method to address unsupervised domain adaptation
(UDA) in a practical setting of continual learning (CL). The goal is to update the model on …

Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation

B Prasanna, S Sanyal, RV Babu - 2023 IEEE/CVF Conference …, 2023 - ieeexplore.ieee.org
In this paper, we propose to develop a method to address unsupervised domain adaptation
(UDA) in a practical setting of continual learning (CL). The goal is to update the model on …

Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models

D Wagner, I Baumann, K Riedhammer… - arXiv preprint arXiv …, 2024 - arxiv.org
This paper explores the improvement of post-training quantization (PTQ) after knowledge
distillation in the Whisper speech foundation model family. We address the challenge of …

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

Z Li, H Xu, T Wang, S Hu, Z Jin, S Hu, J Deng… - arXiv preprint arXiv …, 2024 - arxiv.org
We propose a novel one-pass multiple ASR systems joint compression and quantization
approach using an all-in-one neural model. A single compression cycle allows multiple …

Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations

S Zaiem, T Parcollet, S Essid - arXiv preprint arXiv:2306.00481, 2023 - arxiv.org
Self-Supervised Learning (SSL) has allowed leveraging large amounts of unlabeled speech
data to improve the performance of speech recognition models even with small annotated …

[PDF][PDF] Joint On-Demand Pruning and Online Distillation in Automatic Speech Recognition Language Model Optimization.

S Seo, JH Kim - Computers, Materials & Continua, 2023 - cdn.techscience.cn
Automatic speech recognition (ASR) systems have emerged as indispensable tools across a
wide spectrum of applications, ranging from transcription services to voice-activated …

Stable Distillation: Regularizing Continued Pre-Training for Low-Resource Automatic Speech Recognition

A Seth, S Ghosh, S Umesh… - ICASSP 2024-2024 IEEE …, 2024 - ieeexplore.ieee.org
Continued self-supervised (SSL) pre-training for adapting existing SSL models to the target
domain has shown to be extremely effective for low-resource Automatic Speech Recognition …

[PDF][PDF] Mitigating Overfitting in Structured Pruning of ASR Models with Gradient-Guided Parameter Regularization

DH Kim, JH Chang - isca-archive.org
Recent advancements in automatic speech recognition such as Wav2vec 2.0 and Whisper,
confront deployment challenges due to their substantial model parameters. Model …