Recent work has proposed explicitly inducing language-wise modularity in multilingual LMs
via sparse fine-tuning (SFT) on per-language subnetworks as a means of better guiding …

Pre-trained language models achieve superior performance but are computationally
expensive. Techniques such as pruning and knowledge distillation have been developed to …

Large multilingual language models typically share their parameters across all languages,
which enables cross-lingual task transfer, but learning can also be hindered when training …

Large-scale pretraining requires vast amounts of text in a given language, which limits the
applicability of such techniques to a handful of high-resource languages. Therefore …