Continual pre-training is the paradigm where pre-trained language models (PLMs)
continually acquire fresh knowledge from growing data and gradually get upgraded. Before …

Pre-training-fine-tuning has recently become a new paradigm in natural language
processing, learning better representations of words, sentences, and documents in a self …

Why can pre-trained language models (PLMs) learn universal representations and
effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically …