Editing factual knowledge in language models
The factual knowledge acquired during pre-training and stored in the parameters of
Language Models (LMs) can be useful in downstream tasks (eg, question answering or …
Language Models (LMs) can be useful in downstream tasks (eg, question answering or …
Balancing discriminability and transferability for source-free domain adaptation
Conventional domain adaptation (DA) techniques aim to improve domain transferability by
learning domain-invariant representations; while concurrently preserving the task …
learning domain-invariant representations; while concurrently preserving the task …
Deduplicating training data makes language models better
We find that existing language modeling datasets contain many near-duplicate examples
and long repetitive substrings. As a result, over 1% of the unprompted output of language …
and long repetitive substrings. As a result, over 1% of the unprompted output of language …
From lazy to rich to exclusive task representations in neural networks and neural codes
Neural circuits—both in the brain and in “artificial” neural network models—learn to solve a
remarkable variety of tasks, and there is a great current opportunity to use neural networks …
remarkable variety of tasks, and there is a great current opportunity to use neural networks …
Fast machine unlearning without retraining through selective synaptic dampening
Machine unlearning, the ability for a machine learning model to forget, is becoming
increasingly important to comply with data privacy regulations, as well as to remove harmful …
increasingly important to comply with data privacy regulations, as well as to remove harmful …
Subsidiary prototype alignment for universal domain adaptation
Abstract Universal Domain Adaptation (UniDA) deals with the problem of knowledge transfer
between two datasets with domain-shift as well as category-shift. The goal is to categorize …
between two datasets with domain-shift as well as category-shift. The goal is to categorize …
Feddefender: Client-side attack-tolerant federated learning
Federated learning enables learning from decentralized data sources without compromising
privacy, which makes it a crucial technique. However, it is vulnerable to model poisoning …
privacy, which makes it a crucial technique. However, it is vulnerable to model poisoning …
Sparse double descent: Where network pruning aggravates overfitting
People usually believe that network pruning not only reduces the computational cost of deep
networks, but also prevents overfitting by decreasing model capacity. However, our work …
networks, but also prevents overfitting by decreasing model capacity. However, our work …
On memorization in probabilistic deep generative models
G van den Burg, C Williams - Advances in Neural …, 2021 - proceedings.neurips.cc
Recent advances in deep generative models have led to impressive results in a variety of
application domains. Motivated by the possibility that deep learning models might memorize …
application domains. Motivated by the possibility that deep learning models might memorize …
Task-aware information routing from common representation space in lifelong learning
Intelligent systems deployed in the real world suffer from catastrophic forgetting when
exposed to a sequence of tasks. Humans, on the other hand, acquire, consolidate, and …
exposed to a sequence of tasks. Humans, on the other hand, acquire, consolidate, and …