Recyclable tuning for continual pre-training
Continual pre-training is the paradigm where pre-trained language models (PLMs)
continually acquire fresh knowledge from growing data and gradually get upgraded. Before …
continually acquire fresh knowledge from growing data and gradually get upgraded. Before …
[PDF][PDF] Pre-trained Models for Representation Learning
Pre-training-fine-tuning has recently become a new paradigm in natural language
processing, learning better representations of words, sentences, and documents in a self …
processing, learning better representations of words, sentences, and documents in a self …
Exploring Universal Intrinsic Task Subspace for Few-shot Learning via Prompt Tuning
Why can pre-trained language models (PLMs) learn universal representations and
effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically …
effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically …