Scaling laws are useful guides for developing language models, but there are still gaps
between current scaling studies and how language models are ultimately trained and …

Previous language model pre-training methods have uniformly applied a next-token
prediction loss to all training tokens. Challenging this norm, we posit that" Not all tokens in a …

Research on scaling large language models (LLMs) has primarily focused on model
parameters and training data size, overlooking the role of vocabulary size.% Intuitively …

Though Large Language Models (LLMs) have demonstrated the powerful capabilities of few-
shot learning through prompting methods, supervised training is still necessary for complex …

Scaling law principles indicate a power-law correlation between loss and variables such as
model size, dataset size, and computational resources utilized during training. These …

Scaling laws have driven remarkable progress across machine learning domains like
language modeling and computer vision. However, the exploration of scaling laws in …

The interest in linear complexity models for large language models is on the rise, although
their scaling capacity remains uncertain. In this study, we present the scaling laws for linear …

Comprehensively understanding and accurately predicting the performance of large
language models across diverse downstream tasks has emerged as a pivotal challenge in …

Deep learning-based speech enhancement (SE) models have achieved impressive
performance in the past decade. Numerous advanced architectures have been designed to …

Transfer learning is a machine learning (ML) paradigm where performance on a desired end
task 1 is improved by exploiting” knowledge” from other tasks. The technique has become a …