In this work we identify the dormant neuron phenomenon in deep reinforcement learning,
where an agent's network suffers from an increasing number of inactive neurons, thereby …

Pretrained language models (PLMs) are today the primary model for natural language
processing. Despite their impressive downstream performance, it can be difficult to apply …

Forgetting is often seen as an unwanted characteristic in both human and machine learning.
However, we propose that forgetting can in fact be favorable to learning. We introduce" …

To overcome the imbalanced multimodal learning problem, where models prefer the training
of specific modalities, existing methods propose to control the training of uni-modal …

The simulated annealing algorithm aims to improve model convergence through multiple
restarts of training. However existing annealing algorithms overlook the correlation between …

Re-initializing a neural network during training has been observed to improve generalization
in recent works. Yet it is neither widely adopted in deep learning practice nor is it often used …

Cross-domain few-shot learning (CD-FSL), where there are few target samples under
extreme differences between source and target domains, has recently attracted huge …

The proliferation of algorithms and commercial tools for the creation of synthetic audio has
resulted in a significant increase in the amount of inaccurate information, particularly on …

Recent advances in deep learning for physics have focused on discovering shared
representations of target systems by incorporating physics priors or inductive biases into …

This work identifies a simple pre-training mechanism that leads to representations exhibiting
better continual and transfer learning. This mechanism—the repeated resetting of weights in …