The llama 3 herd of models
Modern artificial intelligence (AI) systems are powered by foundation models. This paper
presents a new set of foundation models, called Llama 3. It is a herd of language models …
presents a new set of foundation models, called Llama 3. It is a herd of language models …
Fine tuning vs. retrieval augmented generation for less popular knowledge
Language Models (LMs) memorize a vast amount of factual knowledge, exhibiting strong
performance across diverse tasks and domains. However, it has been observed that the …
performance across diverse tasks and domains. However, it has been observed that the …
[PDF][PDF] A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and …
Large language models (LLM) have demonstrated emergent abilities in text generation,
question answering, and reasoning, facilitating various tasks and domains. Despite their …
question answering, and reasoning, facilitating various tasks and domains. Despite their …
Training on the test task confounds evaluation and emergence
We study a fundamental problem in the evaluation of large language models that we call
training on the test task. Unlike wrongful practices like training on the test data, leakage, or …
training on the test task. Unlike wrongful practices like training on the test data, leakage, or …
Resolving discrepancies in compute-optimal scaling of language models
Kaplan et al. and Hoffmann et al. developed influential scaling laws for the optimal model
size as a function of the compute budget, but these laws yield substantially different …
size as a function of the compute budget, but these laws yield substantially different …
Open-Ethical AI: Advancements in Open-Source Human-Centric Neural Language Models
This survey summarises the most recent methods for building and assessing helpful, honest,
and harmless neural language models, considering small, medium, and large-size models …
and harmless neural language models, considering small, medium, and large-size models …
Skywork-reward: Bag of tricks for reward modeling in llms
In this report, we introduce a collection of methods to enhance reward modeling for LLMs,
focusing specifically on data-centric techniques. We propose effective data selection and …
focusing specifically on data-centric techniques. We propose effective data selection and …
A survey on efficient inference for large language models
Large Language Models (LLMs) have attracted extensive attention due to their remarkable
performance across various tasks. However, the substantial computational and memory …
performance across various tasks. However, the substantial computational and memory …
Empirical guidelines for deploying llms onto resource-constrained edge devices
The scaling laws have become the de facto guidelines for designing large language models
(LLMs), but they were studied under the assumption of unlimited computing resources for …
(LLMs), but they were studied under the assumption of unlimited computing resources for …
Preference tuning with human feedback on language, speech, and vision tasks: A survey
Preference tuning is a crucial process for aligning deep generative models with human
preferences. This survey offers a thorough overview of recent advancements in preference …
preferences. This survey offers a thorough overview of recent advancements in preference …