Using Language Models on Low-end Hardware
arXiv preprint arXiv:2305.02350, 2023•arxiv.org
This paper evaluates the viability of using fixed language models for training text
classification networks on low-end hardware. We combine language models with a CNN
architecture and put together a comprehensive benchmark with 8 datasets covering single-
label and multi-label classification of topic, sentiment, and genre. Our observations are
distilled into a list of trade-offs, concluding that there are scenarios, where not fine-tuning a
language model yields competitive effectiveness at faster training, requiring only a quarter of …
classification networks on low-end hardware. We combine language models with a CNN
architecture and put together a comprehensive benchmark with 8 datasets covering single-
label and multi-label classification of topic, sentiment, and genre. Our observations are
distilled into a list of trade-offs, concluding that there are scenarios, where not fine-tuning a
language model yields competitive effectiveness at faster training, requiring only a quarter of …
This paper evaluates the viability of using fixed language models for training text classification networks on low-end hardware. We combine language models with a CNN architecture and put together a comprehensive benchmark with 8 datasets covering single-label and multi-label classification of topic, sentiment, and genre. Our observations are distilled into a list of trade-offs, concluding that there are scenarios, where not fine-tuning a language model yields competitive effectiveness at faster training, requiring only a quarter of the memory compared to fine-tuning.
arxiv.org
以上显示的是最相近的搜索结果。 查看全部搜索结果