Bloom: A 176b-parameter open-access multilingual language model

T Le Scao, A Fan, C Akiki, E Pavlick, S Ilić, D Hesslow… - 2023 - inria.hal.science
Large language models (LLMs) have been shown to be able to perform new tasks based on
a few demonstrations or natural language instructions. While these capabilities have led to …

Baichuan 2: Open large-scale language models

A Yang, B Xiao, B Wang, B Zhang, C Bian… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) have demonstrated remarkable performance on a variety of
natural language tasks based on just a few examples of natural language instructions …

Megaverse: Benchmarking large language models across languages, modalities, models and tasks

S Ahuja, D Aggarwal, V Gumma, I Watts… - arXiv preprint arXiv …, 2023 - arxiv.org
Recently, there has been a rapid advancement in research on Large Language Models
(LLMs), resulting in significant progress in several Natural Language Processing (NLP) …

Aya model: An instruction finetuned open-access multilingual language model

A Üstün, V Aryabumi, ZX Yong, WY Ko… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent breakthroughs in large language models (LLMs) have centered around a handful of
data-rich languages. What does it take to broaden access to breakthroughs beyond first …

Llm360: Towards fully transparent open-source llms

Z Liu, A Qiao, W Neiswanger, H Wang, B Tan… - arXiv preprint arXiv …, 2023 - arxiv.org
The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon,
and Mistral, provides diverse options for AI practitioners and researchers. However, most …

Glot500: Scaling multilingual corpora and language models to 500 languages

A Imani, P Lin, AH Kargaran, S Severini… - arXiv preprint arXiv …, 2023 - arxiv.org
The NLP community has mainly focused on scaling Large Language Models (LLMs)
vertically, ie, making them better for about 100 languages. We instead scale LLMs …

Deepseek llm: Scaling open-source language models with longtermism

X Bi, D Chen, G Chen, S Chen, D Dai, C Deng… - arXiv preprint arXiv …, 2024 - arxiv.org
The rapid development of open-source large language models (LLMs) has been truly
remarkable. However, the scaling law described in previous literature presents varying …

Datasets for large language models: A comprehensive survey

Y Liu, J Cao, C Liu, K Ding, L Jin - arXiv preprint arXiv:2402.18041, 2024 - arxiv.org
This paper embarks on an exploration into the Large Language Model (LLM) datasets,
which play a crucial role in the remarkable advancements of LLMs. The datasets serve as …

Polylm: An open source polyglot large language model

X Wei, H Wei, H Lin, T Li, P Zhang, X Ren, M Li… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) demonstrate remarkable ability to comprehend, reason, and
generate following nature language instructions. However, the development of LLMs has …

Large language models: A survey

S Minaee, T Mikolov, N Nikzad, M Chenaghlu… - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) have drawn a lot of attention due to their strong
performance on a wide range of natural language tasks, since the release of ChatGPT in …