In this work we identify the dormant neuron phenomenon in deep reinforcement learning,
where an agent's network suffers from an increasing number of inactive neurons, thereby …

In this work, we investigate the implicit regularization induced by teacher-student learning
dynamics in self-distillation. To isolate its effect, we describe a simple experiment where we …

Commonsense reasoning based on knowledge graphs (KGs) is a challenging task that
requires predicting complex questions over the described textual contexts and relevant …

This work identifies a simple pre-training mechanism that leads to representations exhibiting
better continual and transfer learning. This mechanism--the repeated resetting of weights in …

Deep neural networks (DNNs) are often trained on the premise that the complete training
data set is provided ahead of time. However, in real-world scenarios, data often arrive in …

The first layer of a deep CNN backbone applies filters to an image to extract the basic
features available to later layers. During training, some filters may go inactive, mean ing all …

This study investigates the loss of generalization ability in neural networks, revisiting warm-
starting experiments from Ash & Adams. Our empirical analysis reveals that common …

To overcome the imbalanced multimodal learning problem, where models prefer the training
of specific modalities, existing methods propose to control the training of uni-modal …

In this work, we show that simultaneously training and mixing neural networks is a promising
way to conduct Neural Architecture Search (NAS). For hyperparameter optimization, reusing …

Feedback-driven greybox fuzzing is one of the cornerstones of modern bug detection
techniques. Its flexibility, automated nature, and effectiveness render it an indispensable tool …